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ABSTRACT 

This paper proposes a welcome hypothesis: a computationally simple device is sufficient for processing 
natural language. Traditionally it has been argued that processing natural language syntax requires very 
powerful machinery. Many engineers have come to this rather grim conclusion; almost all working parsers 
are actually Turing Machines (I'M). For example, Woods specifically designed his Augmented Transition 
Networks (A'I'Ns) to be Turing [Equivalent. If the problem is really as hard as it appears, then the only 
solution is to grin and bear it. Our own position is that parsing acceptable sentences is simpler because there 
arc constraints on human performance that drastically reduce the computational requirements (time and 
space bounds). Although ideal linguistic competence is very complex, this observation may not apply directly 
to a real processing problem such as parsing. By including performance factors, it is possible to simplify the 
computation. We will propose two performance limitations, bounded memory and deterministic control, which 
have been incorporated in a new parser YAP. 
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1. Introduction 

'This paper proposes a welcome hypothesis: a computational simple device 1 is sufficient for processing 
natural language. Traditionally it has bcefl argued that processing natural language syntax requires very 
powerful machinery. Many engineers have come to this rather grimcowclusjonialmost all working parsers 
arc actually luring Machines (VM). 2 For example, Woodsi^cc»fica% designed his Augmented Transition 
Networks (ATNs) to be Turing Hqtnvaknt. 

( 1 ) "It is well known (cf. [Chomsky(>4]) that the strict context-free grammar model is not an adequate 
mechanism for ctetraclcrking the subtleiics^natural languages n „Whcp conditions and actions 
arc added to the a«s, the model attains the power of a l^n* macliinc, although the basic 
operations which it performs arc natufaT ones for language analysis. Using these conditions and 
actions, the modeHs toapublc of pcrfocaung tte cqwivalcat of transformational analysis without 
the need for a separate transforaiational«<mpt>«cBt."4\Voods30} 

If the problem is really as hard as it appears, then the only solution is to grin and bear it. Our own position is 
that parsing acceptable sentences is simpler because there arc constraints on human performance that exclude 
all the "harder" cases. A mil parser can take advantage of these perfofmancc constraints (e.g. limited 
memory) so that it can be simpler and more efficient flian Woods' iii£aj model which is designed to parse the 
entire competence grammar. 



1 Throughout this work, the complexity notion will be used in its computational sense as a measure of time and space 
resources required by an optimal pmeessor. The term will not be used in the linguistic sense (i.e. the size of the grammar 
itself) l» general, one can trade one off for the other, which leads to considerable confusion. The mc of a program 
(linguistic complexity) is typically inversely related to the power of the interpreter (computational complexity). This point 
is discussed more thoroughly in chapter 6. :• 

-> li is important' In distinguish amjwtuiiwuil complexity (time and space bounds) from computational class (finite stale 
FS context free CF. context sensitive CS. luring machine TM). A grammar thai describes a large class is generally more 
difficult u> process lhan a more lighlly constrained grammar.' For example; FSgiammars can Ik parsed. with bounded 
space- all others consume unbounded space. Similar comments probably hold for lime complexity, too (ihough the proof 
is an open problem.) That is. FS grammars can be parsed in linear time, whereas CF grammars probably require more 

lime in Ihe worst case. 

3 In fairness to Ihe ATN and Transformation.] ' .... mar. it should be noted that (here have been efforts to reduce the 
generative capacity. For example. Kaplan (personal coimmH.ication), [Wu*J.s73) and [Peters and Ritch.e73] discuss 
various restrictions to assure decidability. Unfortunately, this niOvq is not sunitjenl to guarantee efhcient (e.g. polynomial 
lime) processing: parsing decidable grammars may be effective, but it is hardly effkim. 
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1.1 The Competence/Performance Dichotomy 
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The approach crucially depends on BcitaaB££ constraint* to shrink the search space of possible derivations. 
Formerly engineers such as Woods attempted to rmxlcl aaDC^fiflSfi -without .pcrfimrwncc constraints, and not 
surprisingly, they found they needed inordinate resources to . do .so.. We suggest that a real processor 
incorporates both competence (grammar) and performance (lime and space* constraints. Hence it is possible 
to build a small efficient processor by exploiting die performance model. Ibis is particularly clear from 
Chomsky's original description of the performance/competence dichotomy. 

(2) "Linguistic theory is concerned primarily with an ideal speaker-listener, in a completely 
homogeneous speech-community, who knows its krogOagc perfectly and is unaffected by such 
grammatically irrelevant conditions as memory Mmkariom, distractions, shifts, of attention and 
interest, and errors (random or characteristic) in applying his knowledge of the language in 
actually performance.... We thus make a fundamental distinction between competence (the 
speaker-hearer's knowledge of his language) and performance (the actual use of language in 
concrete situations)." [Chomsky65, pp 3-4, italics added] 

The proposed model is more efficient and more restrictive than Woods' ATN. It is more efficient because it 
doesn't have the resources to waste and it is more restrictive because it doesn't have the resources to explore as 
many possibilities. Kor example, there are some sentences which will require a very long time on an ATN; 
our model wilt reject these sentences as unacceptable (not in die performance model) because it doesn't have 
the time to figure it out. We believe there arc two reasons for rejecting sentences; a sentence may be 
ungmnunalical (excluded from competence) or it may be unacceptable (excluded from performance). 4 The 
term acceptable was coined by Chomsky to refer to: 

(3) "... utterances that arc perfectly natural and immediately comprehensible without papcr-and- 
pcncil analysis, and in no way bizarre or outlandish ...The more acceptable sentences are those 
that arc more likely to be produced, more easily understood, kssclumsy, and in some sense more 
natural. The unacceptable sentences one would tend to avoid and replace by more acceptable 
variants, wherever possible in actual discourse." [Chomsky65, pp. 10J. 



4 This position should be distinguished from Kaplans hypothesis (personal communication) that the processing 
grammar is identical to the competence grammar. We suggest that there are some extra-grammatical factors (e.g. memory 
limitations) which distinguish the two. 
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Acceptability is assigned independently from grammaticamy; the four logical possibilities arc illustrated by 
(4). 5 

(4) It is raining. 

#Tom figured that that Susan wanted to take the cat out bothered Betsy out. 
♦They aw running. r 

*#Tom and slept the dog. 

Chomsky formulated this distinction in order tt» separate wrotevtmt processing constraints (e.g. limited time 
and space) from the grammatically questions which he has been studying. Our hypothesis that a simple 
device can process language, is then, by definition, a hypothesis about die performance model. Acceptability 
judgments will bear crucially on the matter. 

The problem is to design a parser that approximates competence with realistic resources. Unacceptable 
sentences should be excluded because they require mordlnaWrcartirccs to process' ungrammatical sentences 
should be rejected because Uicy violate competence idealizations (o^^ Hie design 

criteria arc summarized below; 

(5) What arc some reasonable performance; approximation*? j. 

(6) How can they be implemented without sacrificing linguistic generalizations? 

1.2 The FS Hypothesis 

We will assume a severe processing limitation on available short term memory (STM), as commonly 
suggested in the psycholinguistic literature ([Frazicr79J, [Frazicr and Fodor78], [Cowpcr76], [Brcsnan78J, 
[Kimball73, 75}. (ChomskyGll). technically a machine with limited memory > a finite state machine (FSM) 
which has very nice computational properties when compared to a* arbitrary TM,. Most importantly, a FSM 
requires less time and space in the worst case, 'flicrc are siwm; .other advantages which we have not explored 



5. These examples arc taken from [Kiuiba«73J. A bash murk (#) is used to indicate unacceplabilily; an asterisk (♦) is 
used hi the traditional fashion ki denote ungrammalicalily. . .; . . 

6 Just as Chomsky ideali/ed panwiiaueality from ,Wm unexplained irrelevant factors, it will be useful to idealize 
acceptability. In this work, we arc nuaMnteresicd in time and sp^fchavjor in th^ limit as sentences grow; we will not 
address borderline cases wheiv jndgnwnts lend to he extremely* ^wWc. this move is often taken in complexity 
arguments which study limiting growth, but ignore constants (bqtdedine cases). 
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The FS Hypothesis IJ 

in detail. For example, it is easier to run a FSM im-evcrse. ITiis may haw some important implications if one 
were attempting to build a single model for both production and generation as suggested in [Kay75]. 

When discussing certain performance issues (c.g. ccntcr-cmbcdding), 8 it will be most useful to. view .the 
processor as a FSM; on the other hand, competence phenomena (e.gi subjacency) 9 suggest a more abstract 
point of view. Because of a lack of I'M resources, the processor cannot literally apply rules of competence; 
rather, it resorts to more computationally realistic approximations. Whenever a Competence idealization calls 
for inordinate resources. -there will be a discrepancy- between the competence idealization and its performance 
realization. 



1.2.1 Center-embedding 



Chomsky and Bar-Hillcl independently showed that (arbitrarily deep) center-embedded structures require 
unbounded memory [Chomsky59a,b] [Bar-Hill P 161J [l.angcndocn75]. As predicted, ccntcr-cmbcdding is 
cly compromised in performance; it quickly becomes unacceptable, even at relatively shallow depths. 



sever 



(7) #[Thc man [who the boy [who the students recognized] pointed out] is a friend of mine.] 

(8) #[Thc rat [the cat [the dog chased] bit] ate the cheese.] 



7 



/ Trivially all physical machines are F$Ms. The FS hypothesis is interesting. Ihough. because the memory limitation is 
so severe (i e two or three clauses) that u is a crucial issue in many pnititeil situatioiw.-SiiiHlar comments can be made 
about modern computers. Most ..engineers would model a W»l ^e ^ipMte^sy^em as a TM However, it would be 
hard to think of a computer as a TM ifil had only I bit of memory. How much memory docs .1 lake before a FSW is bes 
modeled as a TM? The answer may depend oil the current price o*4n*morjr. oWhitl once seemed unreasonable, may not 

be so unrealistic today. . . 

X. A Lcuer-embcddcd sentence contains an embedded clause surrounded by lexical material from the higher clause. 

[. \[. ])]. where both x and v contain lexical material. 

5 iW*r 's « formal linguistic notion whicb constrains the applicability of a transformation. (Informally, subjacency 
is a |.K-:iliiv principle: all transformations must be local to a single cyelk »K<e (^^clatffie) or to two adjacent cyclic nodes.) 
We offer subjacency as an example of a competence idealization. In general.- though, it isextremely difficult tqprove that 
a particular phenomenon is necessarily a matter of compelertee: We have no proof that subjacency .s a competence 
universal, and similarly, we have no proof that center-embedding w a praee^ miiversal. Our assessments are most 
plausible, ihough conceivably, they might be incorrccL < 
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A memory limitation provides a very attractive account of center-embedding phenomena (in the limit). 10 

(9) "This fact [that deeply center-embedded sentences arc unacceptable], and this alone, follows torn 
the assumption of fineness of memory (which no one, surely, has ever questioned). 
[Chomsky61,pp. 127] 

1.2.2 Respectively 

What other phenomena follow from a memory limitation?' Center-embedding is the most striking example, 
but it is not unique. There have been many refutations of KS competence models; each one illustrates the 
point: computationally complex structures are unacceptable. Consider toe respectively construction wh,ch ,s 
notorious for its crossing dependencies. 12 As predicted, it too becomes rapidly unacceptable. 

(10) John and Jack knew Tim and Mike, respectively. 

'.'John, Jack and Sam knew Tim, Mike and Rob, respectively. 

??John. Jack, Sam, and Tom knew Tim, Mike, Rob and Hill, respectively. 

wjohn. Jack Sam, and Tom knew Tim. Mike Rob and Bill, respectively. 

1.2.3 Lasnik's Noncorefcrence Rule 

Lasnik's noncorefcrence rule [I.asnik76] is another source of evidence. 13 The rule observes that two noun 
phrases in a particular structural configuration arc noncorcfcrential. 



and complement clauses. . , . .^a^u cimoostcd that 

1 1 [ft, -I Iillcl61] argued thai respectively proves the competence model is not CF. I has b en w, I s fcgcsK I i « 
sSely is really' semantic issue which shouldn't concern syntax. AM*.*!. ti» rxmU us we I tak ^* « 
Z2f anions constructions (Dutch verbs IJ luybregtsjo,. Swedish ^^^^^^^ 
pose the same problem. If all of these arguments are mistaken and grammar s .n fact only CI . then .1 .s even 
defend the FS Hypothesis. (Only center-embedding would haw to be excluded.) hr „ oWl(l , 

2 Crossing dependencies are beyond CF comply, The proof use, the pumprng '™" l"'^ *> we 

V I. cm be .miied that this rule is not a syntactic rule and Hence it is inrfcvanl to the FS hypolheMs. Actually, we 
bdie'l 1,1 Uk I K hypo,hesis is more general; it applies at levels of linguistic preceding, not just the syntacUc 
component. 



I.aviik '.n Xoncoreference Rule - 14 ■ Section 1.2.3 

(11) The Noncorcfcrcncc Rule : Given two noun phrases NPj, NP 2 in a sentence, if NPj precedes and 
commands 14 NP 2 and NP 2 is not a pronoun, then NP^ and NP 2 arc noncorcfcrcntial. 

lor example, each John in (12) must refer to different people, since the first John both precedes and 
commands the second. This rule lias unbounded consequences; it applies even when there are an arbitrary 
number of clauses between NP, and NP->. Consequently, unbounded memory is required to process the rule; 
it becomes harder and harder to enforce as more and more names arc mentioned. His rule is part of a 
competence model; in performance, it seems necessary to approximate the rule. As the memory requirements 
grow, the performance model is less and less likely to establish the noncorcfcrcntial link. In (12), the 
co-indexed noun phrases cannot be preferential. As the depth increases, the noncorcfcrcntial judgments 
become less and less sharp, even though (12)-(14) arc all equally imgrammatical. 

(12) *#l)id you hear that Johiij told the teacher that John- threw the first punch. 

(13) *V.\ )id \ou hear that Johiij told the teacher that Bill said that Johiij threw the first punch. 

(14) *?l)id you hear that Johiij told the teacher that Hill said that Sam thought Johiij threw die first 

punch. 

Ideal rules of competence do not (and should not) specify real processing limitations (e.g. limited memory); 
these arc matters of performance. (12)-(14) do not refute I.asnik's rule in any way; they merely point out that 
its performance realization has some important empirical differences from I.asnik's idealization. 

1.2.4 Conturgencc 

On ihe other hand, there arc idealizations which can be realized in performance without approximations. For 
example it seems that movement phenomena can cross unbounded distances without degrading acceptability. 
Compare this with the center-embedding and respectively examples previously discussed. 



1-}. Inn urn. ills. ,i phrase precedes phiascs lo its right. For example, a precedes y in: ... x ... y ... A phrases commands 
pina-cs in -uhoidinalc ilauses. That is, ,v commands each y in: [ ... x ... [ ... y | ... [ s ...\j ... See [I.asnik7(>] for more 

discussion. 

i\ S'nik i' '"iinaiiis lepuri that the\ noticed iioncorcfercncc, but chose lo ignore it in the more complex cases. This 
m. cms i' i -.-id tiki with i uir account that it is loo difficult to establish the noncoreference links. 

\'i V\ c ..\pihi I ii-cd the same verbs to illustrate the recursive nature of these constructions. The) would be more 

• ■ .'iiar: kail. '-'.Acpia! T. it" \ : used d life rent verbs. 
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(15) There seems to seem ...to be a problem. movrnp 

(16) Whiit did Bob say that Mil said that ... John liked? move-wh 

Wc claim that ccntcr-cmbcdding and respectively demand* trnbountfctf rt&niVccs whereas movement has a 
bounded cost (even in the worst case). 17 Wc will argue in chdpiCfc 5V6 and 8 that a machine can process 
unbounded movement with very limited resources. Movement phenomena (unlike ccntcr-cmbcdding) can be 
implemented in a performance model without approximation. It is a welcome result when performance and 
competence happen to converge, as in the movement case; there will be no empirical differences between the 
idealization and its realization. However, there is no logical necessity that performance and competence will 
ultimately converge in every area. The FS hypodicsis, if correct, would necessitate compromising many 
competence idealizations. 

1.3 The Proposed Model: YAP 

Some psycholinguists believe there is a natural mapping from ideal competence onto a realistic processing 
model. This hypothesis is intuitively attractive, even though there is no logical reason Biat ft need be the 
case. 19 Unfortunately, the psycholinguistic literature docs not precisely describe a mapping which is 
consistent with our FS hypothesis. 20 Wc have implemented a parser (YAP) which behaves like a complex 
competence model on acceptable cases, but fails to parse more difficult unacceptable sentences. Ibis 
performance model looks very similar to the more complex competence machine on acceptable sentences 



17. The human processor may nut be optimal. The functional argument observes that an optimal processor could 
process unbounded movement with bounded resources. This should encourage further invesligalion. but it alone is not 
suffieieiuevWeneethmihehum;mprocC!»»rh;«fHi^alpii8p^rti«i .:> j 

We claim movement will never consume more than a bounded cos^; U^cost,,^ independent of. the length of the 
sentence. Some movement sentences may be easier' than others. For example, there is considerable experimental 
evidence suggesting that subject relatives (a) arc easier than object relatives (b). • ■■■■ 

(a) I saw the boy who liked you. 

(b) I saw the boy who you liked. 

However, we believe the difference between (a) and (b) is independent of their lengths. 

18. We have given only three examples: cemer-emb«fdcjing, cn«sing(denvr^ei)vies and iioi»ec)reference although there 
are many more. .Ccntcr-cmbcdding and crossing depaulencivs : wer« f |iHei»d«d i« be illustrative of stntctural limitations; 
noncoreference is typical of interpretive rules (such as pronominal binding). 

19. Chomsky and Lisnik (personal communicalbn) have each suggested U\*U, the competence model might generate a 
non-computable set. If this were indeed the case, it wonk! eem .lutlikuj)} lhal-lhW' COU'dbe a mapping. 

2(1. Chart parsers (such jis GSP [Kaplan73]) do not satisfy our requir^nwnus for a psyeht)k)gieally realLslic mapping since 
they are inconsistent with our FS hypothesis. It is not deaf. how chart parsers can account fiy die evidence in favor of Die 
FS hypothesis. 
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even though it "happens" to run in severely limited memory. Since it is a minimal augmentation of existing 
psychological and linguistic work, it preserves their accomplishments, and in addition, achieves computational 
advantages. Chapter 2 will discuss the particular augmentation which allows YAP to conserve memory, and 
hence reduce complexity to that of a l-'S machine. 

The basic design of YAP is similar to Marcus' Parsifal [Marcus79], with an additional limitation on memory. 
His parser, like most stack machine parsers, will occasionally fill the stack with structures it no longer needs, 
consuming unbounded memory. To achieve the Unite memory limitation, it must be guaranteed that this 
never happens on acceptable structures. That is, there must be a "forgetting" procedure (like a garbage 
collector)- for cleaning out the stack so that acceptable sentences can be parsed without causing a stack 
overflow. Everything on the stack should be there for a reason; in Marcus' machine it is possible to have 
something on the stack which cannot be referenced again. Equipped with its forgettcr, YAP runs on a 
bounded stack even though it is approximating a much more complicated machine (e.g. a PDA). 22 

1.4 Closure Strategies 

The forgetting (closure) notion is crucial to this thesis; it enables YAP to parse unbounded structures with 
only finite memory. - There arc two closure procedures mentioned in the psycholinguistic literature: 
Kimball's early closure [Kimball73, 75] and Fra/icr's late closure [Krazicr79] [Krazicr and Fodor78]. We will 
argue that Kimball's procedure is too ruthless, closing phrases too soon, whereas Frazicr's procedure is too 
conservative, wasting memory. Admittedly it is easier to criticize than to offer constructive solutions. 
Chapter 2 will develop some tests for evaluating solutions, and then propose a compromise which should 
perform better than cither of the two extremes, early closure and late closure, but it will hardly be the final 
word. The closure puzzle is extremely difficult, but also crucial to understanding the seemingly idiosyncratic 
parsing behavior that people exhibit. 



?1. I ho "garbage collation" analogy is not completely accurate. Garbage collectors return storage to the system when it 
is known that u cannot be referenced again: closure procedures return storage when is it suspected that it will not be 
referenced again. 



?2. A push down automaton (PDA) is a formalization of unbounded slack machines. 



?..l Mounded memory was the original motivation for closure. Some closure formulations are heuristic; they close a 
phrase before it is known that the phrase in question cannot be referenced again. Theoretically, though, closure need not 
be hciuistic: u is possible for a FSM to parse non-center-embedcled CF structures without heuristics. We have opted for a 
heuristic formulation which appears to more practical (as we will argue in the next section). 
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t.5 Marcus' Determinism Hypothesis 

live memory constraint becomes particularly interesting when it is combined with |.a control constraint such as 
Marcus' Determinism Hv pothcstsJ Man:us791. llic Determinism Hypothesis claims that once the processor is 
committed to a particular path, it is extremely difficult to select an alternative. Kor example, most readers will 
misinterpret the underlined portion of (17)-(19)i and men have cpiisjdcjraplg ^jfTlculty continuing. Kor tliis 
reason, these unacceptable sentences arc often called Qaftdcn P a i^f lGP). A incmory limitation alone fails to 
predict the unacceptability of (1 7)-( 19) because el's don't ccnicr-cjnbcd vcjy, deeply (and hence there exits a 
FSM which could parse these GP sentences). Determinism offers an additional constraint on memory 
allocation which provides an account for the data. 24 

(17) it Hie horse raced past the barn fell. 

(18) #Jo hn lifted a hu pdredffo^pd baas. 

09) # I told the hpv pie dog bit Sue would help him. 

There have been many other attempts to capture the same intuitive motion. Kimball's Processing Principle 
[Kimball73], McDonald's Indclibity Stipulation [McDonald79], and Krazicr's "shunting" notion 
[Frazicr and l ;r odor78] arc typical examples from the psycholinguists literature, 'lhc "shunting" notion 
assigns a high cost to backing up past a phrase mat has been "shutttCcf* from one stag& to another. 

(20) Indclibiljtv : "Once a linguistic decision has been made, it cannot be retracted -- it has been 
written with 'indelible ink' ... It requires every choice made during trie production process, at 
whatever level, cannot be changed once it has been fnatfe- choices mlhtljc made correctly the 
first time." {McDonald79, pp. 16J 

(21) " Principle Seven (Processing) : When a phrase is closed, it is pushed down into a syntactic 
(possible semantic) processing stage and cleared fferrishbitnerm memory." fKhrtbaH73 pp. 39| 



24. There arc oilier possible accounts which may be very similar 10 Mamis" account. For example, GPs arc often relaled 
to backup in non-delerminislic frameworks. HoWcvcr. u i.n nut clewr how such an account can distinguish backup on aGP 
from backup on an acceptable sentence. One solution places a Ixminl on backup to enable the parser to backup on the 
acceptable sentences but not on GPs. fn some sense: tlm is very similar to Marcus? approach: he provides a bound on 
lookahcad (analogous to bounded backup) which dtslingutsh« GPs fh» acceptable sciences. 
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Marcus' Determinism Hypothesis " i° 



Although the "determinism" notion is widely discussed in the literature, it is extremely difficult to describe 
precisely. At first we believed the memory constraint alone would subsume Marcus' hypothesis, thus 
providing a precise independently motivated account. Since all FSMs have a deterministic realization, 25 it 
was originally supposed that the memory limitation guaranteed that the parser is deterministic (or equivalent 
to one that is). Although the argument is theoretically sound, it is mistaken. 26 The deterministic realization 
may have many more states than the corresponding non-deterministic FSM. These extra suites are extremely 
costly and lack empirical justification. They would enable the machine to parse GPs by delaying the critical 
decision. 27 In spirit, Marcus' Determinism Hypothesis excludes encoding non-determinism by exploding the 
stale space in this way; it assumes that most exploded states arc not reachable in performance. This amounts 
to an exponential reduction in the size of the state space, which is an interesting claim, not subsumed by FS 
(which only requires the state space to be finite). 



The forgetting procedure, which is the subject of chapter 2, will be "deterministic"; it will not backup or undo 
previous decisions. Consequently, the machine will not only reject deeply center-embedded sentences but it 
will also reject sentences such as (22) where the heuristic forgetting procedure makes a mistake (takes a garden 
path). 

(22) # Harold heard [that John told the teacher [that Bill said that Sam thought that Mike threw the 
first punch] yesterday]. 

Marcus' Determinism Hypothesis predicts that some sentences would be garden paths (since die state space 
cannot be exploded), but it alone docs not identify which sentences are GPs and which ones are not. He 
proposes a specific parsing model (Parsifal) to identify garden paths. Parsifal makes a single left to right pass 
over the sentence. It has to decide what to do at each point based upon a limited lookahcad of three 
constituents. According to Marcus, certain sentences require more lookahcad to disambiguate 
alt- .r!th.nia.lly. and consequently, Parsifal has to guess what to do. In the garden path case, Parsifal guesses 
incorrectly. 



?5. \ nomdclenninislie FSM with n slates is equivalent lo another deterministic FSM with 2 states. 
~>h i mi mdcbled to Ken Wc.xlcr for pointing this out. 

?7. I he .Nplodu! Mates encode disjunctive ailernalives (as observed in [Swarloul7X]). Intuitively. GPs suggest that it 
isn't no'.sihl. lo dekn the erilieal decision: the machine has to decide which way to proceed. 

v; m-.r-.is- Inpothesis is necessarily vamie because there is no clear way lo distinguish an exploded stale from a 
primitive .(../without reference to a particular machine (grammar). The definition becomes more precise when state 
a>siiT.iikiii ■ ex iiukpoidenllj motivated (by linguistic generalizations). 
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The three constituent limit is a very good description; all the garden path sentences shown above would 
require a four constituent lookahead to disambiguate correctly, (23) illustrates Marcus' account on a typical 
GP. It would be acceptable if the machine looked ahead another constituent 

(23) # The horse [ L raced] [ 2 past] [ 3 the barn] f 4 fell]. 

The three constituent story is not a complete explanation. Why does Parsifal guess that mm/ is a main verb 
and not a participle? The main verb interpretation is apparently the unmarked (preferred) case. Would it be 
possible to have a language where the participle reading was die unmarked case? 

1.6 Fra/.icr's Principles 

Fra/icr [l ; razicr79] [Frazicr and Fodor78} has attempted to describe the unmarked interpretations. She has 
proposed two principles which arc presumably universal. TTibrc is rtnirrtuitivc Rinctional motivation for these 
principles; they appear to require fewer resources (memory and backup) than the alternatives. 'Fra/icr has 
provided considerable experimental evidence as empirical verification. 

(24) M inimal Attachment : Attach incoming materia) into tlic phrase marker 1 being constructed using 
the fewest nodes consistent with the weH-fenaedness miles of the language. 

(25) I .ate Closure : When possible, attach incoming material into the clause or phrase currently being 
parsed. 



29. In practice the lookahead will consist of noun phrases and single words: thu machine docs not, ft»r example, build 
prepositional phrases in the lookahead buffer. Unfortunately this is crucial to Marcus" account of the 01' phenomena; 
Parsifal does rtof analyze sentence (23) as. The Itorsef/ ntc&t] [^pdst Uie bamj fj/efljl \f il did. then ii would be able to 
disambiguate lite sentence. 

There are some other problems with this account. For example, material after the third constituent shouldn't affect 
the judgments, and yet. the sentences below seem to be more acceptable than (23). 

The horse raced past U the barn] fell down. 
The horse raced past [-j the barn] stumbled. 

We have no explanation for this data. Nevertheless. Marcus' account is the best description in the literature: wc will 
accept it for the lime being despite its problems. 
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Frazicr's position is basically compatible with Marcus'; her principles define the unmarked actions when 
there is insufficient lookahead to be certain. Late Closure, which is relevant to the discussion on forgetting, is 
central to chapter 2; Minimal Attachment is the topic of chapter* There i arc some (rare) cases where the 
principles fail to find a correct parse on the first pass, forcing backup in her non-deterministic framework. 
These will be interpreted as marked "counter-examples" in our deterministic FS framework. 30 We will add a 
few marked rules to cover the exceptions. 

1.7 Capturing Generalizations 

Having laid out the basic framework (limited memory and determinism), it is worthwhile to gain some 
breadth. YAP has encoded a competence model strongly resembling the recent work of Brcsnan and Kaplan 
[Hresnan78], (Hrcsnan80], [Kaplan and llrcsnangOJ. They use tw represcB^tions: a constituent structure and 
a functional structure, The former deals with mother/daughtw relationships whereas the latter is concerned 
with grammatical roles (subject, object, etc.) and syntactic features (case, tense, person number, gender, etc.) 
Chapter 3 discusses the YAP implementation of eonstieuent structure, and Chapter 5, the functiona* structure. 

With the Bresnan-Kaplan representation system, it is relatively stFaigrtfforwaird to implement many of their 
analyses. Chapter 6 presents some typical lexical rajJes (raising and passive), thus capturing many of the 
generalizations which were once believed to be beyond the capabilities of a FSM. 

YAP also shares many properties with Parsifal: it is possible to implement Parsifal -style transformations in 
YAP. Chapter 7 implements auxiliary inversion and imperative using Parsifal's approach. This demonstrates 
an alternative method to capture the generalizations that were used to "refute" the FS hypothesis. 

There arc two classes of transformations which have been traditionally problematic for processing: 
wh-movement and conjunction. Chapters 8 and 9 present the approach taken in YAP. Conjunction is 
particularly interesting because it has never been implemented in a Marcus style deterministic parser. Doth. of 
these constructions pose many difficult problems; only some of these have been solved. However, YAP has 
produced some exciting initial results, correctly parsing die following sentences: 



30. She is crucially assuming a non-deterministic framework where the processor can backup past certain errors. In our 
framework, we need some exceptional rules to prevent the processor from taking the Wrong path in the first place. 
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(26) Which boys and girls went? 

(27) Which boys and which girls went? 

(28) Which boys went to the ball and took the jar? 

(29) Which boys went to the ball and to the jar? 

( 30) What boy did Bill look at and give a ball to? 

(31) Bob gave Bill a ball and John ajar. 

(32) Bob saw liill and Sue Mary. 

(33) I want liill. Bob, and John to be nice. 

1.8 Limits of This Research 

It has not been possible to study all issues relevant to parsing; we have touched on just a few of the many 
interesting problems. 'ITiis section will mention some areas for further study. 

(34) Coverage 

(35) Semantic Interaction 

(36) I .cngth Bias (word count) 

(37) Lexical Ambiguity 

1.8.1 Linguistic Coverage 

'lTicrc are many constructions which will not be discussed; YAP is similar to Marcus' Parsifal in coverage. 
Both parsers handle a range of fairly difficult phenomena, arc intended to handle robustly all interactions 
among these phenomena, though neither parser has extensive coverage. YAP docs not parse (38)-(39), for 
example. 

(38) I am taller than Bill. wnporative 

(39) The duck is too old to eat lough movement 

Wc have nothing to say about die internal structure of noun phrases such as (40). it would have been 
relatively straightforward to replicate Marcus' approach. 
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(40) a nice man 
a fallen leaf 
*a given child 

a hundred pound bag 
# a hundred pound bags 

1.8.2 Semantics 

YAP docs very little semantic processing. For example, YAP does not distinguish between animate and 
inanimate objects; (41) and (42) arc equally parsablc from YAP's point of view. 

(41) I gave Bill a ball. 

(42) I gave a ball Bill. 

It is somewhat difficult to distinguish semantics and syntax. YAP docs check grammatical relations (subject, 
object, etc.). (43) and (44) arc correctly distinguished because go and see take different arguments. 

(43) I saw Bill. 

(44) *l went Bill. 

We have not considered bound anaphora and quantifier scope as illustrated below. 

(45) Bill saw himself. bound anaphora 
*Himsclf was seen. 

*Fach other were seen. 

(46) Bill saw everyone. quantifier scope 
F, very one was seen by Bill. 



1.8L3 Length Bias and lexical Ambiguity 

ITicrc arc at least two other processing variables that seem to be relevant: length and lexical ambiguity. Both 
of these are extremely difficult problems which have been widely studied elsewhere [Fra/icrand Fodor78] 
[Mi!nc78a,78b,79,80]. (47) provide some evidence that length (i.e. number of words) influences parsing 
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strategics; 31 (48) illustrates some problems with lexical ambiguity. 

(47) # The woman the man the girt loved met died, length 
771'hc very beautiful young woman the man the gid loved met pn# crujsc ship ii» Maine died of 
cholera in 1962. 

Joe brought the book for Susan. 

Joe brought the book that 1 had been trying to obtain for Susan. 

(48) They were flying planes. lexical ambiguity 
The pupils were small. 

I love building bj&kjs. 

Whatever they arc building blocks the view. 

All of these issues arc extremely important topics for further research. 



31. This evidence is from [Fra/icr and F<*lor78]. Much of il is highly amtroversial: there may be alternative accounts. 
Nevertheless, even if we can't provide adequate evidence, it is most plausible thai length influences parsing strategics. 
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2. Closure 

YAP is essentially a stack machine parser like Marcus' Parsifal wfth an additional bound on stack depth. This 
chapter will deal with die stack allocation problem. Thcr* Wif f be a furgcttmg procedure to remove finislicd 
phrases from die stack so the space can be recycled. ITic procedure will have to decide (hcuristically) when a 
phrase is finished (closed). 

2.1 On I .eft/ Right Biases 

If wc are going to count stack dcpdi, we should be very careful that stack depth corresponds to something 
meaningful. Wc will assume stack space ought to be correlated with die depth of center-embedding. 
Kmpirically, both left and right branching arc relatively free in comparison with center-embedding as 
(49)-(51) illustrate. 32 



32. This position is somewhat different from the "hold hypothesis" [Kaphin75] which accounts for center-embedded 
relative clauses but no other types of center-embedding. We believe that all forms of ccnte i -embedding become rapidly 
unacceptable even at shallow depths. For example, the following scnu-nces from [Rich75] are unacceptable even though 
there are no center-embedded relative clauses. We accept Rich's argument that the "hold hypothesis" fails to account for 
all of the center-embedding facts. 

(a) # I think [claiming [voting Republican] is immoral] is silly. 

(b) # I think claiming [the dog [that bit the burglar] is scared] is silly. 

Notice that both left and right branching have many "bunched up" brackets, l-angendoen (personal communication) 
has observed thai "bundled up" brackets are redundant, and hence they can be deleted without loss of information. In a 
sense, the l-SM manipulates the resulting representation. 

Alternatively, one might view the brackets as corresponding to stack instructions. An open bracket ([) is analogous to 
"push" and close (]) is analogous to "pop". Deleting brackets corresponds to optimizing stack operations (e.g. tail 
recursion [Stecle7.S]). lust as "bunched up" brackets suggest a redundancy, a sequence of "pop" instructions in the logical 
flow of a program indicates wasted slack space. 

One can view closure as deleting brackets, like tail recursion, to optimize stack usage. In left and right branching, it is 
possible to delete the "hunched up" brackets, and hence, bound the maximum slack depth. This fails to bound memory 
requirements in center-embedded cases where there are no "bunched up" brackets to delete. Chomsky's proof 
[Chomsky59a.b] is a formalization of this intuition; center-embedding cannot Ik- optimized because it requires 
unbounded memory (there is no way to -convert a siridtly ' OF grahimaf'.into a F§ graiifinar). On the other hand, it is 
possible to optimize non-ceiucr-cmbeddttfslrucUr^^^ '■ 
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(49) [[[[Thcnian)'s oldest brother's best friendTs^sisterl ... left 

(50) #[ Ihc man |who the boy [who the student iSeoghfeGfl pomfeiwufl tea frterid of thine] center 

(51) [The students recognized the boy [who poii*^iklk^te%i^{#ft^S'aWHc^d oPiHirtcffl right 

Although wc consider left and right branching structures to be equally easy to parse, there have been 
psycholinguistic models with a left/right bias. Kor example, Yngvc [Yngvc60] suggested that left branching 
was more difficult than right branching because left, ^r^ji^|jjpTg ,is^cx||^iT^:ty, ^jl^cult for y lcft.-,U>-rpght 
top-down parser. 33 However, the dual argument could h^e. been used to^ argue against right branching 
structures because they would be costly for a left-to-riqht l ^ toi n^u p parser. Theoretically, neither left nor 
right branching requires unbounded memory because Chomsky ^^ t s^o^^ ,th^t iwu-cc^tenicmbcddcd'CF 
structures (e.g. left and right branching and combinations thereof) could be processed with a finite state 
machine [Chomsky59a,b]. On the other hand, center-embedding is provably difficult because it requires 
u nbounded memory; it cannot be processed by a FSM.^* ^ ;■< ; - ■' ^ «• - 

It is possible that human processing is not optimal in this way|ithcre mq^Uiin fact be a left/right bias even 
though there is no computational motivation. The research strategy taken here will investigate the optimal 
methods first. Although computationally optimal procedures arc not necessarily the ones people do in fact 
use, they arc likely candidates for further research. 

One might argue as Yngve has, that English has a lertfrighf bias tften though ho one has found a 
computational motivation. In fact, it is very difficult tb fihtf acceptable ^ branching clauses in English. 
There doesn't seem to be an acceptable left branchtag$a^ph%s£bf $2j tit Rh^Hsli, as (53) and (54) itluscfate. 
Yngvc's left/ right processing bias is certainly not universal to all languages because there arc languages 
(c.g. Japanese) where left branching is just as productive as right branching is in Enghsh. Hence the. left/right 
asymmetry in English is language specific; it docs not indicate a bias in the human processing system. 5 



33. For example, left branching is " infinitely difficult ^impossible) for an l.l{k) parser. It also caused the Harvard 
Syntactic Analyzer (HPA)[Kuno(>6] considerable problems. 

34. There have been arguments for a fcft/fight asymmetry based oh the assumption that human processing is online 
(lefl-to-right). Chomsky's 1959 proof shows thailhcse arguments are invalid. 

35. We k now of no languages which have botbieff and right branching clauses. This gcnerali/alion is unexplained if it is 
indeed universal. 
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(52) It is interesting that it is indeed true that John likes Mary, right branching 

(53) #That that John likes Mary is indeed true is interesting. 

(54) # John's liking Mary's being indeed true is interesting. 

2.1.1 Kuno's Account 

Why do clauses tend to branch toward the left in Japanese and toward the right in Hnglish? Although there 
arc no known explanations, Knno [Kuno72J |Kuno74J provides a very attractive functional account of a 
related phenomenon: fGrccnbcrg63) noticed -that VSO 3 * languages arc prepositional (right branching) and 
SOV arc usually postpositional (fell branching). (Kuno's account docs not apply in SVO languages like 
Knglish.) 37 

(55) iinjvcisal 3: Languages with dominate VSO order are always prepositional. 

(56) Universal 4: With overwhelmingly greater than chance frequency, languages with normal SOV 
order arc postpositional. [Grocnbcrg63] 

Kuno observed that Grecnbcrg's principles happen to be optimal; if a language violated Grcenbcrg's 
principles then it would be more prone to center-embedding and consequently more difficult to process. 
Consider the case of relative clauses. Kuno observed that relative clauses should precede the head noun 
phrase in SO V languages and follow the head in VSO language in order to avoid center-embedding. This is 
very easy to demonstrate. Hxamplcs (57) and (5$ obey Kuno's obscuvatioivan^ avoid center-embedding; all 
violations do in fact ccntcr-cmbed as (59) and (60) illustrate. 38 

(57) [S 2 2 V 2 that] S| Oj Vj not center-embedded 

(58) V 1 S 1 1 [thatV 2 S 2 2 ] 

(59) S J [thatS 2 2 V 2 ]0 1 Vj center-embedded 

(60) V 1 S 1 ^28202^3110! 



36. S. V and O stand for subject, verb and object. A VSO language .lias the pa'doniinaMl word order: verb, subject, 
object. j, 

37. [Fra/ier80| generalizes Kuno's argument to apply lo SQV language, though her assumptions are somewhat more 
open to dispute. 

38. Recall that a center-embedded clause has lexical material oil both sides of it,. In this case, the ceincr-embcdded 
clauses are surrounded by an S and an O. 



Kuno s Account - 27 - Section 1 1. 1 

Furthermore, notice the complementizer that falls between the head noun phrase and the relative clause. 
This also happens to avoid center-embedding. The alternative would bracket the relative clause between the 
head noun phrase and the complementizer, forcing coiucremfecdding. »Henc<i, aimpicmcrttizcrs will precede 
relative clauses in VSO languages (universal 3) arid' follow relative clauses in S0V languages (universal 4). Ity 
avoiding center-embedding in this way, we have converged on sowkf Of GrccribCfg's principles. fKunb74] 
shows how this reasoning can be applied to some other constructions. 

This does not explain why languages arc this way, but it is an attractive account which should motivate further 
research to verify Grccnbcrg's empirical observations. Furthermore! KiH*"* argument has no left/right 
asymmetry: only center-embedding is considered costly. It seems that center-embedding is universally 
difficult whereas left/right biases are language specific consequences of thctcrttcr-cnibedding universal. 

2.2 Closure Specifications 

We will assume the stack depth should be correlated with the depth of center-embedding. It is up to the 
forgetting procedure to close phrases and remove them from me stack,' so only center-embedded phrases will 
be left on the stack. The procedure could err in cither oftwb directions; it could he overly aithless, cleaning 
out a node (phrase) which will later turn out to be useful, or it could be overly conservative, allowing its 
limited memory to be congested with unnecessary Information. In 'cither case, the parser will run into trouble, 
finding the sentence unacceptable. We have defined the two types of errors below. 39 We will argue that 
K imball's I uirly Closure is premature and Frazicr's 1 .ate Closure is ineffective. 

(61) Premature Closure : The forgetting procedure prematurely removes phrases that turn out to be 
necessary. 

(62) Ineffective Closure : The forgetting procedure docs not remove enough phrases, eventually 
overflowing die limited memory. 



39. These definitions happen to have a functional flavor. We use the functional notion "machine" interchangeably with 
the algebraic notion "grammar". Our definitions should not be laken literally: we do not mean to imply that there is a 
forgetting procedure in the brain just because it might be convenient. We arc merely suggesting that a forgetting 
procedure is a useful metaphor for modeling the compulation that lakes place. 
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2.3 Kimball's Kariy Closure 

The bracketed interpretations of (63)-(65) are unacceptable even though they are grammatical. Presumably, 
the rcM)t matrix 40 was "closed off before die final, phrase, so Uuit the akcrnaUvc attachment was never 
considered. Kimball is crucially assuming dial closure ispossjble beforcths daughters drcmsclvcs have been 
completely parsed. Imagine that a node corresponds to a collection of pwnfers to its daughters: it is finished 
when all of the pointers arc connected. This docs not require that the daughters themselves be llnishcd. l ; or 
example, the node [ s Joe figured [1]\h finished when a pointer is cstaWtshedto the nodel?] even though the 
contents of (?] remain to be discovered. 

(63) # Joe figured [that Susan wanted to take the train to New York] out 

(64) # I met [the boy whom Sam took to the parkfs friend. 

(65) #Tlicgirlj applied for the jobs [that was attractive]:. 

Closure blocks high attachments in sentences like (63)-(65) ty removing the root node from the stack long 
before the last phrase is parsed. For example, it would close die root clause j^befprc that in <67) and who in 
(68) because the nodes \ comp that] and ^^p who] are not immediate constituents of die .root, The root 
clauses would be frozen in the following configurations: [Tom said S-] 41 inland [Joe looked NP] in (68). 
Having closed the root, it shouldn't be possible to reference it again. In particular, nothing else can attach 
directly to the root. 42 'ITiis model inherently assumes that memory is costly and presumably fairly limited. 
Otherwise, there wouldn't be a motivation for closing ofT phrases. 

(66) Kimbalfs Harly Closure: A phrase is dosed as soon as possible, i.c., unless the next node parsed is 
an immediate constituent of that phrase. [Kimbal!73] 

(67) [ s lorn said 

[ s . that Hill had taken the cleaning out ... yesterday 

(68) [ g Joe looked the friend 



4n. A matrix is roughly equivalent to ji phrase or a clause. A matrix is a frame with slots for a mother and several 
daughters. The j^y. matrix is the highest clause. 

41. We use an x-bar [Jackcndoff77) notation, s- (s bar) dominates .v in embedded clauses (\- -> cotup s). It is also 
important to notice that the s- in from said S-] is not completely finished; it is possible id attach material to the embedded 
x- but not to the closed root. 

42. Kimball's closure is premature in these examples since it is possible to interpret yesterday attaching high as in: Tom 
said filial Bill had taken the cleaning out] yesterday. 



Kimball '* Early Closure -29- Section 2.3 



[ s . who had smashed his now car ... up 

There is a slight problem with Kimball's formulation which will become important when we propose our own 
propositi. Ilic unless clause should have a second' condition to block closure until a phrase has all of its 
obligatory daughters. For example, taking Kimball's definition literally, Sj '(| s Thc boy s 2 ...]) should close 
before who in (69) because who is not an immediate constituent of Sj. ITiis would be a mistake because Sj 
docs not yet have a verb phrase. Closure should wait for all the obligatory daughters. For example, an s has 
two obligatory daughters: a noun phrase and a verb phrase; Consequently, sj cannot close before who 
because it doesn't have its obligatory ycrb phrase. 43 

(69) [ | ' Hie boy [ 2 who the teacher always liked best). did something really awful.] 

2.3.1 A Counter- Example 

Although Kimball's motivation to save stack space is well-founded, the precise formulation makes an 
incorrect prediction. 44 If the upper matrix is jsajh/ closed 0$, then it shouldn't be possible to attach aju£hjng 
to it. Yet (70)-(7 1 ) form a minimal pair where the final constituent attaches low in «nc case, as Kimball would 
predict, but high in the other, thus providing a counter-example to Kimball's story. Hvidcntly, the root was 
closed prematurely in (7 1) because it is possible to attach arottewdwerto ifc 

(70) I called [the guy who smashed my brand new car up]. low attachment 

(71) I called [the guy who smashed my brand new car] a rotten driver. hi$h attachment 



43. A s cun Uikc ;i number of optional adjuncts and conjuncts. 

44. We have a mclhwlological suspicion about any theory which predicts an unexpected asymmetry. Kimball's 
principles (as staled in [Kimball73]) have two such asymmetries; his model is both lop-down and righl associative. It 
happens that his predictions are basically correct for a righl branching language like English, but not tor a left branching 
language such as Japanese (Cowper7(>. pp. 29-31]. Kimball's principles am (laic several phenomena, involving both 
closure and language type. Il ought lo be possible to describe I he closure phenomenon independently of word order. An 
ideal explanation would not distinguish between left and righl. bccaMscsojwe languages arc toft branching and some are 
right branching. This is really a rather minor point though; restating thci feet* in this way should pose no particular 
problems. 
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Kimball would probably not interpret his closure strategy as literally as wc have. Unfortunately computer 
models are brutally literal. Although there is considerable content to Kimball's proposal (closing before 
memory overflows), the precise formulation has some flaws. Wc will reformulate the basic notion along with 
some ideas proposed by Frazicr. 

2.4 I'ra/icr's I, ate Closure 

Suppose that the upper matrix is not closed off, as Kimball suggested, but rather, temporarily out of view. 
Imagine that only the lowest matrix is available at any given moment, and that the higher matrices arc stacked 
up. The decision then becomes whether to attach to the current matrix or to close it off, making the next 
higher matrix available. The strategy attaches as low as possible; it will attach high if all the lower 
attachments are impossible. Kimball's strategy, on the other hand, prevents higher attachments by closing off 
the higher matrices as soon as possible. In (70), according to Fra/icr's late closure, up can attach 45 to the 
lower matrix, so it does; whereas in (71), a wlten driver cannot attach low, so the lower matrix is closed off, 
allowing the next higher attachment. Frazicr calls this strategy late closure because lower nodes (matrices) are 
closed as late as possible, after all the lower attachments have been tried. She contrasts her approach with 
Kimball's early closure , where the higher matrices are closed very early, before the lower matrices arc done. 

(72) Fra/.ier's I .ate Closure : When possible, attach incoming material into die clause or phrase 
currently being parsed. 

2.4.1 A Problem: Right launching 

I .ate Closure is an improvement because it docs not close prematurely like Karly Closure. Unfortunately, it is 
too conservative, allowing nodes to remain open (not closed) too long, congesting valuable stack space. Our 
compromise will modify Fra/icr's strategy enabling higher clauses to be closed earlier under certain marked 
conditions. As late closure is defined by Frazicr, right branching structures such as (73) and (74) arc a real 
problem. 



45. Deciding whether a node can or cannot attach is a difficult question which must be addressed. YAP uses the 
functional structure [Kaplan and liresnanXO] and the phrase structure rules. For now wc will have to appeal to the 

reader's intuitions. 
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(73) This js the dog that chased the cat that ran after thcuW that ate the cheese that you left in the trap 
that Mary bought at the store that .„ 

(74) 1 consider every candidate likely to be considered capable of being considered somewhat less than 
honest toward the people who ... 

Ilic problem is that the machine will eventually fill up with unfinished matrices, unable to close anything 
because it hasn't reached the bottom right-most clause. Hence it will find these right branching sentences just 
as unacceptable as center-embedding. Perhaps Kimball's suggestion is premature, but Kra/icr's is ineffective. 
Ihc compromise solution will strongly resemble Frazicr's late closure strategy except there will be one 
marked case of early closure to handle right branching structures. 



:i,>,rx 



Our argument is like all complexity arguments; it considefsthe iimiting^pchavior as the number of clauses 
increase. Certainly there arc numerous other factors which decide borderline cases (3-dccp center-embedded 
clauses for example). We have specifically avoided borderline cases because judgments arc so difficult and 
variable; the limiting behavior is much sharper. In these limiting cases, though, there can be no doubt that 
memory limitations arc relevant to parsing strategics. In particular, alternatives cannot explain why there are 
no acceptable sentences with 20-dccp center embedded clauses. Ihc only reason is that memory is limited; 
sec [Chomsky59a,b], [Uar-Hillcl61] and [Langcndocn75] for the mathcmaticaUrguments, 

2.4.2 Analogies from UM and l<R(k) Algorithms 

It would help to abstract the closure problem in terms of fbrnial pa/suig algorithms. Among deterministic 
parsing algorithms, I J4k) parsing corresponds^ the cjrriifisjpowbte closing whereas LR(k) corresponds to 
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closing at the latest possible moment. 46 In IJ.(k), the machine decides to close before any of the daughters 
have been attached, whereas an I.R(k) parser decides to close after all of the daughters have been attached. 
Kimball's scheme is not quite as ruthless as I.I.(k); his parser closes after all but the last daughter has been 
attached. Fra/icr's scheme is remarkably similar to I.R(k), where the closure decisions arc made at the last 
possible moment. Karly closing schemes tend to be premature; they cannot parse as many constructions as 
later closing schemes. In particular, I. I.(k) cannot parse left recursive expressions. Later closing schemes tend 
to be ineffective, wasting memory. An I.R(k) parser will push all the input onto the stack in ihc worst case 
fright branching). 47 Closing early reduces the parser's capabilities whereas closing laic increases the memory 
costs. 48 

It might be noted here that Marcus' parser actually behaves very much like an I.R(k) parser in this respect, 49 
and hence, like Kra/ier's scheme. 50 That is, it pushes the entire right-most branch (from the root to the most 
recently read word) onto the stack, so that it never prematurely closes a node as an l.l.(k) parser docs; on the 
other hand, it will often waste stack space like an LR(k) parser. 



46. Recall thiii boih I.L(k) and I R(k) parse CF grammars on a deterministic slack machine (I)PDA). I.l.(k) is purely 
top-down: the machine decides which production to expand (push onto ihe slack) given the moiher and the next k input 
swuhuls. Ihe stack is popped when the next input symbol matches the lop of the slack. This discovers the left-most 
don\ : .l.on (for ambiguous grammars). I.R(k) is the dual; the machine decides which production to reduce (nop off the 
stack) given the noxl k input symbols and the previous state. Input symbols are pushed onto the stack when there are no 
productions to reduce. I.R(k) finds the right-most derivation. LL(k) is a predictive parser because it predicts expansions 
lop-down: I R(k) ,s i, sluji-reduce parser because it decides whether to shift (push an input symbol) or to reduce (pop a 
production off the stack). 

I I (k) are optimal for purely right branching siructurcs: the stack grows infinitely on left branching structures (doesn't 
hah), and linearly with ihe depth for cenler embedding, but is bounded on right branching. I.R(k) parsing is the dual" it is 
optimal for pureh leti branching slruciures where the slack depth is hounded. On center and right branching ihe slack 
depth grows linearly. I.l.(k) is nol as general as I.R(k). bul it is more space efficient when it works In our terms I I (k) 
parsers sutler trom prcmaiure closure whereas I.R(k) parsers may require more memory (ineffective closure) 
47. It memory is cheap ihen I.R(k) is very aiiraciivc. Currently several computer programming languages are parsed 
vMih I k(k) techniques because Ihe memory demands are tolerable. We are assuming that human short term memory 
hmilalions are far loo severe for such extravagances. 

4X. [Kiniball75] makes a similar point, lie offers two compromise points between I I (k) and I.R(k) and shows that the 
corresponding languages are .-ill properly nesied. (ISoth compromises appear to be premature: Ihe arguments are not very 
interesting.) 

49. Marcus himself has argued this point on many occasions (personal comnumicalion). 

51). Man. us had not been Ihinking about the closure issue. Nevertheless, his work forms an interesting data point anions 

the possible closure strategies. 
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2.5 A Compromise 

Wc have designed YAP to close late by default (like l,R(k), [Fswjcr?!?] and, JMarcus7?]) with one marked 
exception to alleviate the memory load (in the right branching fiase).- 1 The marked case of carty closure is 
described by the A-over-A early closure principle. It is very much like Kimball's early closure principle except 
that it waits fOr two nodes, hot just one. For example hi 1(75*}, our principle would close [j that Bill said S2] 
just before the thai in S3 whereas Kimball's scheme would close it just before the that in S2. 

(75) John said [j that Hill Siiid[ 2 that Sam said l^ that Jack... 

( 76) T!i£ A-ovcr-A car |y . cjusur c nfl uc j pt e: Given two, phrases uvthe same category (c#. noun phrase, 
verb phrase, clause, etc.), the higher, closes whcji boj.b jncdigibjc^for Kimball closure. That is, 
(1) both nodes arc in die same category, (2) the next node parsed is not an immediate constituent 
of cither, and (3) the mother and ail obligatory daughters have been attached to both nodes. 

'ITiis principle, which is more aggressive than Fra/jcr late closure, enables the parser to process unbounded 
right recursion within a bounded stack by constantly closing off. However, it is not nearly as ruthless as 
Kimball's early closure, because it .waits for two nodes, which may alleviate the problems that Frazicr 
observed with Kimball's strategy. 

There are some questions about the borderline cases where judgments are extremely variable. Although the 
A-ovcr-A early closure principle makes very sharp distinctions, borderline eases arc often questionable. See 
[Cowper76] for an amazing collection of subtle judgments tiiat confound every proposal yet made. However, 
wc think that die A-ovcr-A notion is a step in the right direction; it has the desired limiting behavior, 
although the borderline cases arc not yet understood. Chomsky comes to a similar conclusion: 



51. Parly closure is very similar to a compiler optimization culled I flil rcc uys jon. which converts right recursive 
expressions into iterative ones, thus opumi/ing.sjLuck usage. .Qin^pilers *Wld ^'fluirni^optinii/iationsonly when they can 
prove that the structure is right rctiirsive; the A-uver:A closure prir^jpk is somewhat heuristic becaiise the structure may 
turn out to be ccntcr-cmbcddcd* 

52. A node can't close tuilil it knows who its mother is. This is impoflwl haause i<, is possible in YAI> to build nodes 
hoitom-up. They might have ^Ulhei; daughters, but not their mother, Secondly, we assume die root doesn't have a 
mother and hence it cannot close. This will have some important iiMpjicatipns as wt? will sec. 

53. Notice that an A-ovcr-A-ovcr-A principle would ..!.'> have the same limjtirtg behavior. In general, if there are n 
categories, an A-over-A principle would limit the stack depth to 2n (in the rijiln branching ease) whereas an 
A-over-A-over-A principle would limit the depth to 3». The difference (between 2 and.,3) is a constant which cannot be 
distinguished by a complexity argument of this sort. It is an einpirical question whkh isprcfcrable. 
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(77) "Obviously, acceptability will be a matter of degree, along various dimensions. One could go on 
to propose various operational tests to specify the notion more precisely (for example, rapidity, 
correctness, and uniformity of recall and rcco^hltiOri! normalcy of intonation). For present 
purposes, it is unnecessary to delimit it more carefully." [Cht>mslty6S pp. I0J 

We arc still experimenting with the YAP system, looking for a more complete solution to the closure puzzle. 54 
2.5.1 Predictions 

Many of Frazicr's observations also apply here because YAP closes late by default as in her model (except for 
the A-over-A early closure principle). As long as A u ovcr*A early dostffc dbesn't apply, YAP behaves just like 
Frazicr's model. In particular, both Krii^r's tifc closure and YAP arc not premature , unlike Kimball's 
scheme. Consider the "counter-example" to Kimball's early closure: 

(78) I called the guy [who smashed my car ... upj 

(79) I called the guy [who smashed my carj... a rotten driver. 

Kimball's scheme prematurely doses the root clause Just before w/w Which is not an immediate constituent of 
the root. ITiat is, it prematurely decides the root looks like [g I called NPJ regardless of what follows who. As 
wc have previously noted, when a rotten driver is finally reached, Kimball's scheme will be stuck. Frazicr's 
late closure is an improvement because it keeps the root open until a rotten driver is parsed. YAP behaves just 
like Frazicr's model in this case, because the A-over-A early closure principle docs not apply. Hence, YAP is 
not premature (at least in this case). 



54. The A-over-A closure principle is an incremental forgclling procedure. One could imagine another type of forgetting 
procedure which waited until the system "an shorttm spnet amj only then ii would search the slack for "garbage". (In 
some sense. Fra/icrs IW avoids "shunting" until it is running short oft Space. Hence the PPP is efleclire. though the SSS 
is now stuck with the problem.) In this framework, the forgcrting procedure acts as a background process which 
"interrupt";" the parser whenever space runs short. This interrupt approach is quite plausible though it poses a few 
problems. First, like a MSP garbage coHeclor which also Waits until (he computer is oil I of CONS space, it is not 
quasi-real-time (bounded amount of time' between reading amy two input symttrts)! This is a particularly undesirable 
property of I. ISP for real-time applications (like airline guidance systems) beejtiisl' the airplane might crash during a 
garbage colleciion. Secondly, interrupt driven systems are extremely difficult lb debug and verify because it is very 
difficult lo replicate the same situation twice. Consequently, it would appear quite difficiilt to model real psychological 
data within an interrupt framework. Thirdly, the interrupt mechanism isyci another device which must be stipulated. 
The incremental approach avoids alt of these technical problems. 
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(80) John said [ 1 that Hill said [ 2 Uiat Sam said [3 that ... 

YAP's closure is more effective in the right branching case because A-ovcr-A early closure will apply. For 
example, pure late closure wttf eventually lead to a memory overflow in right branching sentences like (80). 
Pure late closure will find right branching just as bad as ccntcr-cmbcdding. On die other hand, YAP's early 
closure will constantly close nodes ; early (before reading the entire sentence), thus preventing a memory 
overflow. For example, it will 'close i/flj that Bill said Sft'as stx>n as it attaches the last daughter to s^ In 
sentences like (80), early closure removes nodes just as fast as new ones arc being formed. In this case, YAP is 
effective (unlike Frazicr's late closure). 

(81) John said [ l that Bill called the guy [ 2 that Sam said {3 that ... X 

(82) #John said \ { that Bill called the guy [ 2 that Sam said [3 that ...J] X 

Ilicrc arc some empirical consequences of closing early. For example, nothing tart attach to a closed node. 
Hence it should be possible to test the A-ovcr-A early closure principle by noting whether or not nodes closed 
under the principle actually do block further attachments. For example, in (81) once Sj is closed it shouldn't 
be possible attach X to it as in (82). We will illustrate several types of A"s: adjuncts, conjuncts and optional 
arguments. 

(83) John said [j that Bill called the guy ... yesterday. adjunct 

(84) John said (j d»t Bill called the guy /..awl Saw called Ac girl, conjunct 

(85) John said [j that Bill caflcdtHb gtfy ...a rtktcn driver, optional argument 

Closure principles merely state which attachments arc possible; they do not specify preferences. There is 
considerable literature noting that ATs tend to attach as low as possible. A similar principle will be discussed 
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in chapter 4. It will be qualified to favor attachments to the lowest possible open node. 55 

Ilicrc is a second testable prediction: no interpretive rule car} apply into a closed node. ITiat is, linguistic 
domains (command, c-command, f-cornmand, etc.) 56 have gaping holes where phrases have been closed off. 
These holes arc opaque islands to rules of bound anaphora (reflexive and I reciprocal), 57 quantifier scope, 58 
and reference (noncorcfcrcncc). We will show that ^prediction ; appears to hold for I-asflik's 
noncorcfcrcncc rule. We will not discuss other interpretive rules here. 

2.5.2 Adjuncts 

The underlined phrases in (86) and (87) arc called adjuncts. They can generally attach to any open node along 
the right hand edge, thus accounting for the multiple interpretations. (Admittedly, there is a strong 
preference for low attachments.) 



55. This will attach to the lower matrix even if the higher attachment * the only grtinurauical possibility. For example. 
(a) and (b) arc marginal because the final phrase tends to attach \m, wljich is ungranunatical. : . 

(a) ?l kjoked the guy who sniiished my car ub. 

(b) ?I'nl the block which is on the box on the tab te. 

It seems that this is the correct prediction in the unmarked case: the acceptability might improve if the parser could be 
given some helpful hints (punctuation or intonation breaks) to block the low attachment. Unfortunately, this account 
incorrectly predicts that the sentence will become more acceptable if there is a second argument for the higher matrix as 
in: #1 looked ihc guy [who smashed my car up] up. The second up cannot atUtch to the embedded clause, so it should 
attach to the higher matrix, fulfilling the grammatically requirements. Unfortunately, the sentence is much worse with 
the second up. This is a serious problem for the current approach. 

56. Interpretive rules, such as Lisnik's Noncorcfcrcncc rule, apply over limited domains of the parse tree. We have 
already defined command: c-command and f-command are slight variations. Command is defined in terms of clauses, 
c-command in terms of constituents, and /-command in terms of functional structure (chapter 5). 

57. reflexive: They hit themselves. 
reciprocal: They hit each other. 

58. (a) Fveryonc in this room speaks at least two languages. 

(b) At least two languages arc spoken by everyone in this room. 

Sentence (a) has so-called wide interpretation (for all people there arc two languages such that each person speaks them); 
sentence (b) lias narrow scope (there are two languages such that everyone speaks them). See [Vanl.ehn78|. 
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(86) John said that mil did it Y£5t££dax. 

John said [that Bill did ft ycstertfoy). — ^ low attachment 

John said (that Hiiff did itjyc^rday. ■ high attachment 

(87) John said that Bill did it m gel ahead . 

John said [that Bill did it to get ahead). low attachment 

John said Ithat Hill did it] to get ahead. high attachment 

The interesting claim is that adjuncts cannot attach to closed nodes. For example, yesterday can not attach to 
s 7 in sentences tike (88) because A L ovcr- A early closure wtitild aifplyftrst. 

• '■'■ :'■ .'■•■.■..;:■.• >:f ■. u>'-. '■.-.■■ ■ -it, '.-'■:■■■ . 

(88) John said (j that Bill said that Sam said ... that Jack did it yesterday. 

Although this seems to, be the case, Ujss yep haid fc> test tb<},««nsutucncjs relations with i^mc advcrbials like 
yesterday. Purpose adjuncts (such as to get ahead) suggest a much sharper test. Notice that (89) and (90) have 
difTcrcntiindcfSt(^)d subject, rcflcctirtg^ the atfft 

(89) John said [that Bill did it (for Bill) to get ahead). 

(90) John said [that Bill did it) (for Jotin)Vo get ahead! 

It is possible to test the constituency relations indirectly using well-known tacts about subjects. For example, 
(9l)-(92) arc unambiguous; the alternative constituency relations (93)-(94) arc ungrammatical since they 
violate binding conditions on reflexives. 

(91) They said [that Ml die? it to gdhJoSSlfoutof hotwatfer). 

(92) JM said [that BiH did k)togttibaaias& aWNrf hw w«& . 

(93) *Th£y. said [that Bill did it] to get himself out of hot water. 

(94) ♦'ITicy said [that fijll did it to get themselves out of hot water). 

Now, it should be possible to test whether a node is closed. ITic purpose adjunct in sentences (95)-(98) must 
attach to s,. But this will be unacceptable when s ; is closed as in (98). As usual, the borderline cases (96>(97) 
are somewhat marginal. 
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(95) I )idy on hear [ y they did it to get themselves out of hot water? 

(96) ?i)id you hear [j they said that Bill did it to get themselves out of hot water? 

(97) ??l)id you hear ^ dicy said that Hill said that Sam did it to get $«^*cf<OMlaffcot water? 

(98) # Did you hear [j dicy sjiid that Bill s<iid that Sam said that Jack did it to get themselves out of hot 

water? ■>>;''"'■ ■ ■■'■ '"-.'-) ., 



2.5.3 Conjuncts 

There is a similar argument using conjuncts instead of adjuncts. Assuming that closed nodes cannot be 
conjoined, conjunction should become more and more difficult in (99)-(10l), since Sj is more and more likely 
to close early. 

(99) I saw a boy [j who dropped the delicate model airplanef and who picked it up and began to cry. 

(100) ?I saw a boy [^ who dropped die delicate model airplane l 2 he had so carefully been makiug at the 
school)] and who picked it up and began to cry. 

(101) ??I Siiw a boy [j who dropped the delicate model airplane U he had so carefully been making at 
the school [3 where I went [4 when I was young}]]] and who picked it up and began to cry. 

.■■■■■ 1 -*t' ,, i '. ' ! •«* > ■ 

The claim that conjunction is limited to open nodes is also useful in oajsuig. Suppose that we had an 

algorithm for deciding closure. Then we would know exactly which conjunctions arc possible because 

''■'tii" y ' ''•"* ' ''!»'■ ■- ; ■ ' 

conjunction is permitted between open nodes of the same category. ITiis considerably reduces the 

combinatory explosion of possibilities that has made U so troublesome to parse conjunctions. It is an 

interesting fact that conjunctions, at least their acceptable iotofpfctations, arc i^ycr. rnore; (hap a few ways 

ambiguous, even though non-deterministic parsers (such as ATNs) can often find quite a number of absurd 

possibilities. • * . = 



59. Some open nodes may not permit conjunction because they arc slacked up and hence oul of view until the lower 
possibilities f;iil. The preference for ihc lowest open node will be discussed in chapter 4. 

60. There are some grammalicalily constraints on conjunction which further restrict the possibilities which become 
important lo chapter 9. 
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2.5.4 Optional Arguments 

There is a third argument supporting early closure. Unfortunately the data arc extremely controversial 61 and 
there may be several alternative accounts for the facts. It would not be disastrous for the A-ovcr-A early 
closure principle if the facts happen to fall the other way. Nevertheless, we will give the argument because it 
illustrates the approach, even though the evidence is not as clear as it i 



( 102)-( 105) lest whether v, is open or closed. If it is closed, the optional argument a rotten driver cannot attach 
and 'hence the sentence should be unacceptable. 'ITiis accounts for the judgments in the extreme cases; s^ is 
open' in' (102) and hence (102) is acceptable, whereas s f is closed in (105), and hence (105) is unacceptable. As 
usual, die borderline cases (103)-( 104) arc marginal. 'Ihc A-ovcr-A early closure principle happens to exclude 
these marginal cases here; diis should not be taken too literally. < 

( 102) Did you hear [j that I called the guy [ 2 who smashed the car^a rotten driver? 

(103) ?Did you hear [j tliat I called die guy [ 2 who smashed the car [3 that I bought last year) a rotten 
driver? 

(104) ??Did you hear Uiat I called the guy who smashed the car [3 that I bought last year [4 just after die 
old one needed a new transmission]] a rotten driver? 

(105) #l)id you hear Hurt I called'thcguy who smashed Ac car '3 that 1 bought fasi year {4 just after the 
old one needed a new transmission ( s which wouWbaw cost SiflOffl a roiiew driver? 

This account crucially depends on die optionality of the argument a rotten driver. If it were obligatory, tiien it 
wouldn't be possible to close S/ until a second argumcRt to a*// is found- Asdjhcnee>. early closure would not 
be an adequate account of the data because it caftnotapply to the cra^almifiplx^y:^ 2 Inhere issdttic evidence 
that a rotten driver is optional; our informants report that ( 105) is much better without the final phrase. (This 
judgment is controversial.) 



61. Berwick (personal communication) reports different judgments when the crucial examples were spoken. Our own 
informants were given written lexis. Hudi experiments were informal ; ,i :i . 

(>2. A very plausible alternative is that call is lexically ambiguous; there is a verb call NP as in / called John and there is 
another verb call NP NP as in / called him a name. Assuming that a clause can't be closed until its predicate has been 
disambiguated, early closure cannot apply to Ihc crucial mlrix containing the verb call. And hence, the data may not be 
relevant to the closure issue. One could take this argument to an extreme and say that all verbs are lexically ambiguous 
and cannot be disambiguated until the clause is completely parsed, and hence, early closure would always be blocked. 
Then it isn't clear how right branching sentences could be parsed. The lexical ambiguity argument is very tricky. 
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(106) Did you hear that I called the guy who smashed the car that I bought last year just after the old 
one needed a new transmission which would have cost $100? 



2.5.5 Noncoreferencc 

Lasnik's noncoreferencc rule [I.asriik76j is another source of evidence. Previously we showed that 
noncoreferencc in sentences like (107)-( 109) was less and less likely to apply. In this subsection, we will claim 
thai noncorcfcrcntial links cannot be established into a closed node. Again, the extreme cases arc much 
sharper than the borderline. Noncoreferencc is clearly established in (107) where the crucial clause is open. 
The judgments become less and less sharp as s/ is less and less likely to be open. 

(107) *Did you hear [, that Johnj told the teacher that Johnj threw the first punch? 

(108) ?Did you hear ^ that Johnj told the teacher [^ that Bill said that Johnj threw the first punch? 

(109) Did you hear [j that Johnj told the teacher [ 2 that Ulll said [j Unit Sam thought I4 that Johnj threw 
the first punch? 



2.5.6 Root Clauses 

'Die A-ovcr-A closure principle (unlike Kimball's account) predicts that root clauses have a special status with 
respect to closure. r ITic root clause will never close because it can't have a mother. In particular, this suggests 
that it is always possible to conjoin to the root no matter how many clauses intervene. 

(110) I saw a boy who dropped the delicate model airplane he had so carefully been making at the 
school where I went when I was young and you saw a girl do the same. 

Similarly, this predicts that root clauses can always take adjuncts. However, it docs not predict that optional 
arguments can attach to the root because they arc dominated by a verb phrase which docs have a mother. 
Hence, the verb phrase could close early, blocking optional arguments from attaching. 

( 1 1 1 ) # Joe [ vp figured [that Susan wanted to take the train to New YorkJ ... out 
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2.6 Summary 3 

In conclusion, wc have argued .that a memory limitation r,cdj^*tbc,)Pyc}-aUt«Bicajid space requirements (by 
fiat); the competence model alone carmot achieve sue)} tig^Jbspjyjds, ^lj^p^iit is very difficult to discover 
the exact memory allocation procedure, it sccim.that the d^r^^c^uf^nf)!!. offers an interesting set of 
evidence. There arc basically two c\tflome closure mo^y^ t^ |jt,9faUirc 1 K,iiMbaH's early closure and 
Fra/icr's late closure. Wc, have argued fiv a compromise; portion. the A-over-A early ck»suie;|>nincipJc< which 
shares many advances of both previous proposals without $$nc, of .the. attci^ittdjisadvantages, ,0m principle 
is not without its own problems; the borderline cases arc extremely difficult. It seems that there is 
considerable work to be done. 
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3. Constituent Structure Implementation 

YAP's searches for a mapping from a string of words to a set of g^ammaticat relations (subject, object, etc.) In 
the Brcsnan-Kaplan system [Kaplan and BrCSrtdii^J. the rcsirttihi grammatical relations form a ftihctional 
structure (Structure). Incrc is atf mtcrmediate rcprc^ntaltio^ yild' uSc constituent structure (cstructure) 
which captures structural rctatkms (mother; daufht^.sister; ^<!lc): irhe system has an algorithmic procedure 
for building the fstrticturc from the atfroctnrc 1 . IWehajforlwiW dcScVfbc HowYAVdocs this; this chapter is 
mainly concerned with btrildmg the cstructurt ih'thc first place. 

Ine cstructure has similar counterparts in most other linguistic representations. For example, 
transformational grammar starts with a set of CK base rules and then applies a number of transformations. 
Similarly. ATNs start with recursive transition networks (RTNs) which arc CF equivalent and then add a 
number of augmentations. Brcsnan's cstructure is a CF description. Ine mapping from the cstructure to the 
(structure is analogous to transformations in 1X3 and augmentations in ATNs. 

Hicrc arc interesting differences between all these systems; we have adopted the Brcsnan-Kaplan framework 
because it seems easier (to us) to map it into a practical FS deterministic parser. Kven if 1X3, ATNs, and the 
Brcsnan-Kaplan framework arc all notational variants of one another (which is unlikely), the Brcsnan-Kaplan 
framework might be more useful for our purposes since it is not obvious how to encode the other models into 
a FS deterministic parser. 63 



63. There have been many articles relating ATNs to processing strategics [Kaplan72J [Wanner and Maralsos78J 
[Bresnan78j. All of these require more resources (memory and backup) than wc are willing to allocate. Their approach 
appears to be very difficult: although there was great hope in the early papers, it is very difficult to make further progress. 
McDonald (personal communication) has pointed (Mil Unit traditional ATNs are analogous to PI.ANNKR [Ilcwilt72]; 
boil) replace knowledge with brute force automatic backup. More recent Al problem solving languages (c.g.TMS 
[Doylc7K]) replace notions like automatic backup with dependency directed backup. We see the same trend in language 
processing (e.g. GSP [Kaplan75J) tlxHigh there are many details to be solved. We have avoided many of these difficult 
problems by stipulating FS and Determinism. It seems that the llrcsnan-Kaphm framework is more compatible with 
these stipulations than mtm: general frameworks (which permit non-deterministic side-cffecLs), though it would be 
difficult to formalize this intuition. 
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(112) I am a boy. 

013)[ s l np . I] l vp [ v am] [ np _ [ np Lj ct a] [„ boyllffl 

This chapter will discuss how YAP builds the (Structure. The problem is to map a sentence like (1 12) into a 
structure like (113). The cstructurc is a tree 64 of phrases (nodes). Phrases arc delimited by square 
brackets ([ J) 65 labeled with a category (part of speech). A category has two parts: a major catcgorial feature 
(n, v, a, p) 66 and a "bar" level. YAP uses a three-bar system; there arc nouns (n), noun phrases (np) and np 
bars (np). Similarly there arc verbs (v), verb phrases (vp). and participial phrases (vp-). 67 In all, YAP has 4 
major categorical features which have three bar levels, forming 12 categories. Ilicrc arc 6 other categories: s, 
s-, dct, comp, conj and puncL 6 ^ 

3.1 The Machine State 

YAP has four components taken almost directly from Marcus' Parsifal: ,, 

YAP ParsjfaJ 

(1 14) the input stream the input stream 

(If 1 5) the upper buffer the stack 

(116) the lower buffer the lookahcad buffer 

(117) a deterministic FS control device a grammar of pfotiuctkm rules 

A snapshot of the machine is shown in figure 1. The string " =? = W A 1 4 - = = " is printed between the upper 
and lower buffers. Buffer cells arc filled with nodes ( parsed > phrases) which arc printed in square 
brackets (I ]). 69 Both buffers grow in toward the WALL as thc t machine jKirscs toward the center (the WAN,) 
from both directions (both top-down from die root and bottora-up from die input). The upper buffer 
contains mothers which arc building down to the WALL and the lower buffer contains daughters building up 



64. This condition will be weakened to ccodc structural ambiguity (pseudo-attachment). 

65. 7 hesc are often called phrase markers (or P-imrkers) in the linguistic literature. 

66. n = noun; v = verb; a - adjective; p =* preposition 

67. The "bar" system was first introduced in [Chomsky70J to describe certain generalities between noun phrases and 
clauses (i.e. John's having criticized the book and Mm has criticized the boukl Ste JJackvnik»ff77J for a more current 
reference. The term projection refers to the next higher bar level. For example, np- is a projection of np and np is a 
projection of n. The third bar level is the maximal project «m\w\ YAP), fFhere bavo been proposals for five bur systems.) 

68. s = sentence; s- is ^projection of s: del * detei miner; ttmip = cotuplenienti«ar (/br. /As/. ...); eonj = conjunction; 
punct = punctuation. Tht^catcguries. don't fit the bar pattern wry wall. 

69. Printing isa very expensive; M*k: it requires i scnuebing iUw fri^Ciof *he pafsifd sabjiccs la find- the individual words. 
The parser itself is not permitted to undertake such expensive U«ks; Ttthniiailly the printer is nbt part of trw machine. 
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Fig. 1. A Snapshot 

sentence: I am a boy. 
input pointer: boy. 



up3 
up2 
upl 


U 


downl 
down2 


= =WALI 

tnp- '1 
[ v am] 


down 3 


fdet 3 ' 



the upper buffer 
the lower buffer 



to the WALL. Nodes (parsed phrases, i.e. nonterminals) enter and exit near the WAIL in stack fashion (via 
push and pop operations). That is, upl and downl are the "top" of their respective buffers (stacks). New 
words (terminals) enter the lower buffer from the i "bottom" (downS). 

YAP is deterministic and FS for reasons discussed previously. Hie control device (117) is defined to be 
deterministic. That is, from any machine state there, is exactly one applicable grammar rule; backup is 
absolutely excluded. The PS limitation hasbcen itr^plcrnqnlcd in YAP by truncating the buffers Jo 3 fixed 
length and limiting the size of a bufffercell ft'hc bounds wihe two buffera have not <yot boc» defined. The 
first three cells of each buffer arc referenced so frequently that it is convenient to name them upl, 
up2, .... down3as in figure 1. In fact, the buffers may be longer. 'Hie complexity arguments suggest that they 
should be limited, but it is not clear what the limits should be. w Setting the exact limits (constants) would 
require! considerable psychological experimentation. Ihc length of the upper buffer measures the maximum 
allowable depth of center embedding. The lower buffer measures tnc degree of ibokahcad.* 1 



70. The bound on each buffer is a pamnmer which can be adjusted at runtime. 

71. Clmmsky (personal ummiuiiHiiUtMi) points out that IxMinded fcx&abend might be «quivalcnt lo some sort of 
bounded backtracking. In which case, (he lower buffer could be thtk^hl Uf as measuring Iho degree of backtracking. 
[UllmaiidS] discusses 1 wo rnierestingnftjmwIiRitkws of bounded bsidtWaekiug. ("Boimded r»tfraHc*ism f ' is pitiably very 
siniihir lo bounded backtnicking and Ixiunded lookHboad.) ' 
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The Machine Stale " 4J * 

There are some interesting issues associated with filing the si/.c of nodes. In Parsifal, a node is literally a 
pointer to a subtree (parsed phrase). A YAP nude is an abstraction of the relent features, not the entire tree 
itself. 72 It is important to bound die si/.c of a node in order to prevent encoding unbounded memory into the 
nodes." mis guarantees that any predicate can be tested in a fixed amount of time. Parsifal stores subtrees 
in the stack cells; it could take an arbitrary amount of time to search a subtree for some property (such asdic 
value of a trace.) 74 Similarly, die formal system outlined in Kaplan and Brcshun80] permits two unbounded 
nodes to be unified which also requires unbounded time. 75 Although this is a Uicorctical point, it docs bring 
up some vcrv difficult questions regarding abstraction (inheritance). Which features docs a mother inherit 
from its daughters and which features arc opaque to further inquiry? This question will be studied ui 
chapter 5. 

3.2 Production Rules 

Only one more component is needed before the machine can run. We have to specify a procedure for 
determining which actions to apply next. We will begin by describing a very general technique. 'Ifeno* few 
sections will present some more specific techniques which should cover the most common unmarked cases. 
The more general techniques will be used only in very marked „&***■****.■ We introduce them 
first because it is relatively easy to sec that they arc sufficiently powerful; however, they are so powerful that ft 
is very difficult to combine them effectively into a good structured program (grammar). 

•II* general technique is to use a set of production rules (like Parsifal's grammar rules) which uniquely 
determine the actions to perform. That is, the first applicable production rule is selected. We arc strongly 
depending on Marcus' Determinism Hypothesis; the rules cannot backup or sprout new processes like a 
non-deterministic machine. A sample rule wight be: 



ince* md hence it may require unbounded time lo retrieve a value from a k»ng chain of traces. 

S!f pmpe^ P^des ".niSderablc computational power, that system !% capable of parang CS languages of the 

fomr. AVlKapUitt(pew>iial^)mnuinicatk)n)l- / 
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(118) (dcfrulc attach-subj 

(pattern ( = s) ( = np- - vp)) 
(action (attach))) 

This rule would say, if there is an s (sentence) node just above the wall (up 1) and there is an ///>- followed by a 
vp just below the wall (in downl and down2), then attach the up- (downl) to the s(upl). It is very similar to a 
CF rule of the form: 76 

(U9)s->np-vp... 

The pattern has a limited window; it can only reference the first three cells in each buffer and features 
immediately attached to them. (120) lists the syntax for a pattern. TTicrc arc six predicates 

<"Pl> <down3> associated with upl down3, respectively. 77 If the predicates and the lisp expression 78 

return true, then the pattern "matches". 79 

(120) (pattern 

«up3><up2Xupl>) 
(<downl Xdown2><down3>) 
<lisp expression^) 



76. CF rewrite rules are often viewed as lop-down (generative). Tnw asymmetry is purely a matter of convention; they 
could have been formulated in a bollom-up fashion. Our representation is neutral with respect to top-down and 
bottom-up. 

77. If the pattern contains less than three predicates, the specified predicates apply to the cells closest to the WALL. For 
example. (1 18) applies to upl. downl and down2 because upl is the closest to the WALL from the upper buffer, and. 
downl & down2 are the closest from the lower buffer. 

78. This lisp expression must be side-efteci-frec (cannot change the slate of the machine ,in any way). It is also 
constrained to the first three cells in each buffer and their immediate descendants (by convention). 

79. Although it is useful to think of the ..njHiern matching us a linear search, they are initially inverted (hashed) on <upl> 
and<downl>. In practice, approximately seven patterns are tested before finding aii^Ucl{tlie|csl/maich ratio had been 
4: 1 before certain rules were added. Theoretically it should be possible to do ijnui.ii heller; the lest/match ratio sliould be 
2:1 orbclter. 

The matching procedure deserves much more attention. This may be the proper place to incorporate lexical 
expectation and extremely subtle preference data, which are often taken as evidence for a backup mechanism. Since 
l(x)kahead is analogous to backup, it ought lo be possible to encode these* 'ikis in a fooKulieWrraincwdfk. 
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3.3 Actions ,; ' : 

The grammar rules (the deterministic FS control device) modify the machine state througj} ,# number of 
actions . Iliis section will discuss three primitive actions: attach , predict and close . 'ITicsc basic operations 
appear in most parsers in one way or another, Jn an ATM ute ^responding* oporntiims tfcavcrsc an arc, push 
to a new network., and; pop. from a network, lf>,lv<Ti:lcy's ; aJgorM ) b<nlferley?Ol the corresponding operations are 
scan, predict and complete. Basically, any free traversal algorithm will UuvcthWc corrospwndiwg operations: 

(121) move across from one sister to the next (attach, traverse arc, scan} 

( 122) move down from a mother to the first daughter (predict, push) 

(1 23 ) rm>vc i//> from the 'fast daughter toWmbther ''''"'' (close, pop. complete) 



These actions arc implemented in terms of buffer operations. Recall that both buffers arc building toward the 
WALL: the upper buffer holds mothers looking to>dowfi ^^Nift^^il > W^bdibr^^iHd J wXlL) 
whereas die lower buffer hddsdiiughtcrS loofcihg^tfc^upW^heVs. fHicgrass Islilwhys grtchcr oh the 
other side of the WALL, so to speak.) 80 When a daughter and a mother fUialfy ffiVd each other (attach), the 
daughter is popped from the lower buffer because it is no longer looking for a mother. It is then pushed onto 
tlie upper buffer, to enable it to find some of its own daughters. As we will sec, upl will inherit certain 
features (e.g. person, number, gender. ...) from down! to, reflect me au^chracnt., H^aally, tin: mother and 

Fig. 2, The Attack Action 

Attach pops downl from the lower buffer and pushes it into the upper buffer. 

sentence: I am a boy. 
input pointer: boy. 

before aflej 

[ s l I.U 

= =WALL== [ np .IJ 

[ np .H = =WALL=~ ■'* 

[ v am) [ v amJ ! ; 

tdct a ' ldct a J 



80. In GSP terminology [Kapl;in75], the upper buffer holds producertuad the lower buffer holds consumers. 

81. Daughters can be attached before they themselves arc complete. Thj» is cmcial for curly closure. 
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daughter arc linked together in the output structure. This link is also available to the printer, and hence, upl 
prints differently after the attachment. For example in figure 2, upl is printed as ^J before the attachment 
because it dominates no words, but afterwards, it is printctfas £ I] because it dominates the word /. 

The machine proceeds in a middle out fashion away from fte-WAiJll First, it tries to attjjch down 1 to upl as 
we luivc just seen. If that faiMtstama^^^^ 

For example, supptwe .that YAP finished patfng'tlfettl^W-^ unspecified means, 

leaving die machine slate ready for the predict action as in figure 3. Upl contains a clause (L I]) looking for a 
vp daughter and down! contains a verb [ v am] looking for a vp mother. ITic predict action starts a vp node 
between upl and down 1 to bridge the gap. Hie machine can, now continue by attaching upl to downl just as 
it did in figure 2. 

YAP will continue predicting and attaching uMiJ.it r<^j|^,thc 1 sta|e ! s|)w«fi^V figure 4, At this point, upl is 
complete. The machine will close \ipl by, pqppiflgjt i^d^upj^bwff^^W rcimwngit from memory, 
It cannot take on any more daughters. 

3.4 The Phrase Structure Component 

Marcus' riindtine has a number of production rules like (121 ) to decide Which actions to perform. It would be 
possible .to writc-a complete grammaf in this torni; Ifwe did so, YAPwouto took very much lilre his machine. 
The problem with writing a grammar in production rules is that the performance and the competence 



Fig. 3. The Predict Action 

Predict will start a new node between upl and downl. 

sentence: I am a boy. 
input pointer: . 

before after 

Is « Is'] 

= =WALL== ==WALL 

fv am l [ vp ] 

fdct a l Iv 301 ! 

fn b °y] Id* a] 

I n boyJ 
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Fig. 4. The Close Action 

Close pops the upper buffer, thus removing upl from memory. Nothing more can attach to this np. 



sentence: I am a boy. 
input pointer: 

More alter. 

[ s I am a boy] [ s I am a boy] 

[ vp am a hoy] [ vp amabQyJ 

fnp-aboy] ==WALL=: 

= =WAI.L= = [ nunct .l 

ipunct •' 



punct ' 



components tend to become tangled: it is very difficult to write a good structured program (grammar) with 
such elementary building blocks. [Swartout78], [Shipman78] and (Marcus79, chapter 4] have observed that 
Uicrc arc phrase structure (PS) rules hidden inside Marcus' grammar. Shipman then wrote a PS machine 
which used phrase structure rules to decide when to activate rules. 82 It would be desirable if we could add 
phrase structure rules to YAP so diat it could select the next action in an orderly wa^, The phrase structure 
component should cover most normal unmarked cases; production ruks arc reserved fcr marked c 
A typical YAPsphrasc structure rule is as follows (omitting details): 

(124) (dcf-ps-rulc finite-s s 

(csubj obi (s- np-)) 
(chcad obi (vp))) 



This ps-rule is similar to the two CF rules 



83 



82. Marcus has ;i notion of iiclivc rules. For technical fcasottv wc hate not incorporated this idea explicitly in YAP. The 
notion is in fact implicitly encoded in nodes. (RjiehnodekiMw^whiitAteikKiliinflbis.) 

83. Technically, it is closer to fhc ftrtlowing CF rules, l*Owcver» *♦ iK>ntefinin»b (csubj, chcad....) are always 
non-branching. 

s -> csubj chead 
• csubj -><- ■ 
csubj -Vnp* 
chcad -> vp 
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(125)s->np-vp 
(126)s->s-vp 

In Knglish the nilc says that a finite s 84 has two obligatory daughters; the first is the surface subject (csubj) and 
the second is die head (chcad). The first can be cither an s- or np-, and the second, a vp. This rule could have 
been written as a large number of marked production rule* the ps rules are more perspicuous. For example, 
a single ps-rulc replaces ten of Marcus' rules for parsing auxiliaries. 85 Sec [Shipman78] for a translation 
procedure from ps-rulcs to Marcus' production rules. 

There is a PS pointer associated with each node to indicate what the node is "looking for". A PS' pointer is 
written in dotted rule notation where the dot (.) marks which terms have been parsed (sec figure 5). The PS 
pointer is automatically advanced when a daughter is attached as in the figure. 

YAP will use the PS rules to select the next action. When there arc no applicable marked rules, the 
interpreter tries to apply the PS rules. There arc three possible PS actions: ps-attach, ps-prcdict, and ps-close. 
In YAP they arc implemented as follows: 87 

( 127) ps-attach : If downl can attach to upl, men do so. (figure 5) 

(128) ps-prcdict : If the category of upl's next daughter can be determined, then -predict a node of that 
category. (figure 6 top) 

( 1 29) ps-close : If upl can be closed, then do so. (figure 6 bottom) 



84. A finite clause is tensed, as opposed to an infinitive or participial phrase, 
finite: I am a bov. 

infinite: To be a, boy is tough. 

participial: Iteiim a hoi. I know how nt must fee'- 

Racedi^ lhcbam,*hefa<»rse fidt tike gelling even* 

85. Auxiliary verbs arc "helping" verbs-sut'ta us: be.thiim wtfl«W. <*>.- 

86. Parentheses () denote iirrtk»iial-teim&;r»afektls.lV-(k«otft.g|irf»isjve disjunction, and * is the Klecnc star far arbitrary 
repetition. Brackets have very restrictive distribution since they arc difficult to express within the determinism 
framework. 

87. ps-prcdicl has a lop-down asymmetry which is very unfortunate. To compensate for this deficiency, there a?e quite a 
number of production rules to predict bottom-up. The grammar would be much simpler if the ps-prcdicl rule were more 
symmetric. 
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lig.5. PS Attach 
sentence: I am a boy. 
input pointer: boy. 

[ ] finitc-s -> . csubj chcad 

= =WAIJ.= = 

[ 1] normal-x -> cword . 

[ v am] normal-x -> cword . 

( dct a) normal-x -> cword . 

Ikcausc downl is a possible csubj for upl's finitcs, the default PS-attach rule will attach downl to upl, 
leaving the machine in the following .state. Notice, that Uiq ,PS Bomtcj.a&sociatcd with the s node is 
automatically advanced. 

[ s I] finitc-s -> csubj . chcad 

[ I] normal-x -> cword . 

= =WALL== .;., ., ■. ;,. ..;■> 

[ y amJ normal-x -> cword . 



L . a] normal-x -> cword . 



All these rules arc depended upon the ps pointers; d*c conditionais&anattaeh, can predict, and can close) are 
functions of the ps pointers. These rules arc the defaults which can be over-ruled in the marked case by a 
production rule. By introducing these ps rules wc have greatly reduced the number of marked productions 
rules. ITic current grammar has 12 ps-rulcs and 69 production rules. In practice, the ps-rulcs and production 
rules are executed about equally often. The PS rules were designed to strongly resemble Brcsnan-Kaplao's 
constituent structure component just as the producuon roles ixSjcmWcManais'grtmmar. 

3.5 Ordering PS Actions 

(130) attach 

(131) predict 

(132) close 

The ps rules have an unmarked order (130M132) wfc«heaii b$ ©wswulcd by. a marked production rule. In 
Uic unmarked case, first try to attach downl to upl If that doesn't work out, then try to predict Finally, try 
closing up. Kmpirically, this order seems to require a minimum number of marked rules. It favors attaching 
early (low) and closing late. I .ate closure was discussed in chapter 2; early attachment is the subject of 
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V ig. 6. PS Predict & PS Close 


PS Predict 






sentence: I am a 


boy. 




input pointer: . 






fs 1 ! 




finitc-s -> csubj . ch« 


= =WALL= = 






t v am l 




normal-x -> cword . 


ldct a l 




nornial-x -> cword . 


f n boy] 




normal-x -> cword . 



Since the category upTs next daughter is unique (it must be a vp), the PS-predict rule will start a vp node in 
down 1, as illustrated below. 



Is I] 

= =WAU.= 

fvp] 
[ v am] 

Idet a l 
l n boy] 



finitc-s -> csubj . chead 

normal-vp -> . chead (cobj) (excomp) 
normal-x -> cword . 
normal-x -> cword . 
normal-x -> cword . 



II ■ | . Ji l il > »'. ' 



PS Close 

sentence: I am a boy. 
input pointer: 

[ s I am a boy] 
[ vp am a boy] 
[ np .aboy] 
= =WALL= = 
'punct •' 



finitc-s -> csubj chead. 

norm^-vp -> chead (cobj) . (excomp) 

normal-np- -> (cspec) chead . 

normal-x -> cword . 



Since upl can close, the PS close operation would pop it from the upper buffer, thus removing it from 
memory, so no further attachments can be considered. 



[ s 1 am a boy] 
[ vp am a. boy] 
= = WALL= = 
'punct' 



finitc-s -> csubj chead . 

normal-vp -> chead (cobj) . (excomp) 

normal-x -> cword . 
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chapter 4. The rule ordering would attach JIT as low as possible in structures like (133) because ps-aiiach 
precedes ps-chse. (134M136) illustrate this for adjuncts, conjuncts and optional arguments, respectively. Ilic 
next chapter will compare tills approach with alternatives in die literature. 

( 1 33) John called the guy who called the girl who called ... X 

( 1 34) John called die, guy wJao called, foe girt w&o called. ..yesterday, adjunct 
John called thc»guy; who called «fcc girl wlwqrfted ... to make {himself, herself} fed better. 

( 1 35) John called the guy who called the girl who called ... and said "hello". conjunct 

(J 36) John called the guy who called the girl who called ... a rotten driver. optional arguments 

John called die guy who called die girl who chllcd ... up. 
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4. Attachment Strategies 

What types of information should drive the attachment pfioectf? ^^ aw (oMr basic strategics in the 
literature: 88 

(137) SirucluraJiiiaS [Kimball73, 75], [Kra/.icr and l-odor78], [Marcus79], YAP 

»■«>■ ..Wv p^nUWArc-Orderin. ■ ltt*6t)^^ 

(139)Len£ihmas >Ya/.icr^ 

. „• [Crain79] 

(i40)Scjnimii£Jiias . 

Although there arc valid arguments for each of these, positions, we will concentrate on the structural biases in 

this chapter. YAP can encode die other biases ^nM^^^I^^^ 1 " is P r0vidcd ^ ** 
unmarked case) by the proposed rule ordering (attach, predict, and then close). It appears very similar results 
arc produced by Fra/icr's two principles: minimal aitacjimcjil and Jalfi cjusm*. ITiis idea was inspired by 
[Wanncr79pp. 12] which relates Frazicr's principles to certain ATN actions (traverse arc, push and pop) 
which arc similar to our three primitive actions (attach, predict and close.) 



88. Few papers fit the categories perfectly. For example, we h,ve listed the Sausage Machine m two place* bcauKAta 
some struct! components (minimal attachment and hue closure) and some length biases (Preliminary Phrase Package* 
Similarly, we could have listed the arc-ordering nape* under sever.,, headings because arc-ordering can encode many 
lypes of biases, as [Wanner79] quite correctly notes. Pro , lmin „rv Phrase 

89 We have very little to say about length biases. Fra/iers machine has a front end called the Prehmmary Phmse 
^i;>PP) which segments the input stream into manageable chunks that are "shunted off to the next higher sUge 
(SSS) The PPP has severely limited memory (about six words) and it has little or no ability to communicate with the SSS 
except to "shunt" segmented phrases which it will (almost) never see again. This model makes the interestmg predion 
that preliminary segmentation is subject to length biases. hnitnm-un 

There are a few problems with this proposal. First off. it is not clear how to build a PPP. Purely tattoo, up 
segmentation is extremely difficult in general, unless one is will to form all possible segments (when « jpiubaMy _not 
Fra/iers intent.) Secondly, although the length biases are certainly real at some level to s suggestion hat hey play a 
major role in parsing is extremely controversial. For example. fWanncr79] observes that the length factor does not appear 
to alter the preferred interpretation in the following sentences. 

Tom said that Bill had taken it out yesterday. 

Tom said that Hill had taken it yesterday. 

Tom said that Bill took it yesterday. 

Tom said that Bill died yesterday. 
We will accept Wanner's criticisms of the PPP and his alternative proposal (ordering the actions: attach, predict and I then 
close). The interested reader should investigate his paper for more discussion of the PPP and how .1 relates to his 
proposal. 
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(141) Minimal Attachment : Attach incoming material into the phrase marker being constructed using 
the Fewest nodes consistent with the well-formedness rules of the language. 

(142) Late Closure : When possible, attach incoming material into the clause or phrase currently being 
parsed. [Frazicr79 pp. 76] 

(143) Minimal attachment = attach before predict 

(144) I ate closure = close after predicting and attaching 

If the two analogies, (143) and (144), are correct, then the proposed unmarked ordering of ps rules is a valid 
implementation of Frazicr's principles. Her principles were designed to capture a large number of 
performance phenomena, from a psychological point of view. We will address their feasibility from a 
practical engineering point of view. 

4.1 Minimal Attachment 

Minimal attachment prefers (146) and (149) because they have fewer brackets (nodes). 

(145) The horse raced past the barn (fell). ((Fra/icr79 pp. 27D 

(146) +[ s [ llic horse] [ v _ raced past the barn]] ... fell 

(148) Torn heard the latest gossip about the new, neighbors (wasn't trip), (ShnHaf W ]Fra/ier79 pp. 155}) 

(149) -t-' lorn hea rd [the latest gossip about the new neighbors]. 

(150) -Tom heard []thc latest gossip about the new neighbors] wasn't true]. 

4.1.1 Sensitivity to Phrase Structure Rules 

There is a technical problem with this formulation; minimal attachment is extremely sensitive to slight 
modifications in phrase structure rutcs; it would be more robust if. i| counted limiting growth (like a 
complexity argument), not individual nodes. It js not clear* for example, that her counting argument can be 
used to distinguish the following (Fr«.icr79 pp. 24]. 



90. It is useful to further distinguish the acceptable/unacceptable continuum. The plus symbol (+) is used to indicate a 
more acceptable sentence; minus ( - ) indicates a less acceptable one. 
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(151)Samhit[lhcgirl][withabook] high attachment 

( 1 52) Sam hit [the girl with a book] low attachment 

The first has one fewer node using her phrase structure rules; they have the same number in our analysis. 
These borderline cases arc notoriously difficult; human judgments tend to be unreliable and indecisive. For 
example, [Wales and Toncr76] have found that certain ambiguous structures have little or no bias; both 
possibilities arc about equally probable. 'ITiis fact is not Caplttfcd by most attachment strategic* which draw 
very sharp distinctions. Certainly, both Frazier's minimal attachment and our ordering criteria arc guilty of 
this criticism, later in this chapter, we will discuss a marked rule (pseudo-attachment) to cover the 
ambiguous case. 

(153)[ the girl] [with a book] Frazier's analysis high 

(154)[ np [ np the girl] [ pp with a book]] low 

( 1 55) [ np . [ np the girl]] [ pp . with a book] YAP's analysis high 

(156)[ np .[ np thcgirlH pp .withabookJ] low 



4.1.2 Explanations for Minimal Attachment 

Intuitively, the principle appears to conserve computational resources, although the argument has not been 
completely formalized. JWanncr79] argues that it is gcneraHy more efficient to attach before predicting 
because predictions postulate an additional node which prcsurhabfy involves a certain additional cost. Hence 
it is generally cheaper to order attach before predict. This ordering happens to be consistent (more or less) 
with Frazier's minimal attachment strategy. 

It is very difficult to formalize this argument. Although it is generally cheaper to attach before predicting, 
attaching first isn't always cheakper. For example, there are struct!) rally ambiguous sentences such as 
(151)-(152) where attaching 1 first is no more efficient. Even if there! Were m discrepancies between the 
ordering criteria and Frazier's principles, it isn't clear which explains which. Does the ordering criteria 
explain the minimal attachment principle or the other way around? Nevertheless, there is an interesting 
correlation. Despite its problems, we will accept Wanncr's account as an implementation of minimal 
attachment (and leave the explanation question unresolved). 
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[I'odoi and Frazicr80] suggest another explanation. Suppose the parser builds "several" paths in parallel. 
Ilic first one to finish "dominates subsequent processing". This provides a nice motivation of minimal 
attachment; presumably the most minimal path would finish the "race" first since it constructed the fewest 
number of nodes. Similarly, they could account for the ambiguous case as "a double finish" (although 
I'ra/.icr 'happens to argue that this particular case is unambiguous [Fra7.icr79 pp: 143J). 

One has to be cure-Art with tlic parallel processing account. If it is taken too literally (each derivation has its 
own processor), it would trivialize the attempts to limit backup/Jog&thcad (by substituting hardware for 
backup/lookahcad). Ibcrc ought to be a mechanism for bounding parallelism just as there is a mechanism 
for bounding lookahcad in Marcus-style parsers. (In some sense, backup, lookahcad and parallelism arc all 
very similar.) I'odor and Fra/icr's account would be much more satisfying if they also discussed the 
limitations of the parallelism. 

It has been very difficult to find a deep explanation for the principle because it is heuristic (in our 
framework). Iltcrc arc several cases where the principle can- bo overridden. For example, there arc the 
ambiguous cases just mentioned. Also, it has been argued Hint semantic and pragmatic biases can influence 
the judgments. Furthermore, there appear to be sortie empirical censtructions where the most minimal 
attachment is excluded (by competence constraints) permitting a less rwnwnal attachment. Hksc (rare) cases 
constitute yet another class of exceptions, at least in our framework. 91 It is a heuristic to be applied when 
there arc no reliable clues (semantics, pragmatics, or grammatical constraints). Minimal attachment is not like 
ccntcr-cmbcdding, for example, which is universally unacceptable Ccntcr-cmbcdding can be explained by 
the FS hypothesis; we arc not likely to find a similar explanation for minimal attachment It is a "least effort" 
heuristic (in linguistic terminology, it is a "markedness" principle). Heuristics are generally more difficult to 
explain than univcrsals like ccntcr-cmbcdding. 



91. Actually Fra/ier (personal conuminicaiion) disputes this point Since her machine js non-deterministic, these 
"exceptional" cases arc less prohkiMHlic: her machine simpH takes the roust minimal path first and then bucks up when it 
encounters a dead end. Hence it witt eventually find the most minimal grammatical interpretation. In our deterministic 
framework, wc have marked rtiles to look ahead -for the problematic cases,', w cither framework, though, these exceptional 
cases pose a difficulty for an explanation because it is not clear how one can constrain the backup/lookahcad mechanism. 
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Left Branching Structures " •** " 



4.1.3 Uft Branching Structures 

mere arc some cases where the heuristic is crucial. For example, extreme non-minimal attachment 
(predicting before attaching) fails on a left branching structure such as (157), where it would predict infinitely 
many noun phrases. 92 Although the most extreme non-minimal position is theoretically inadequate, there arc 
many compromise positions which may suffice. For example, a parser could make a few predictions before 
attaching, thus creating slightly mm-minimal structures without the theoretical inadequacies. There is no 
explanation for the must minimal strategy. 

(157)np->np'sn 

(158) John's father's ... brother's friend 

4.2 Garden Paths 

Left branching is an extreme case; Frazier's experimeatt were more concerned with the well-known garden 
path (GP) phenomena such as (159H W2). Thesears caltedGP sentences because the reader is led down the 
garden path so to speak. It would appear that the performance model has optimized the process of 
recognizing the vast majority of sentences which do not contain garden paths so that these GP sentences are 
no longer acceptable. 

(159) #Thc horse raced past the barn fell. 

(160) # r l be ship floated on the water sank. 

(161) # John lifted a hundred pound bags. 

(162) #1 told the boy the dog bit Sue would help him. 



92 Some parsing models in ihc literature dually have .his prob.cm. For example the UXk) algorithm^ which predicts 
before m.chinu will infinitely predict on left bunching structures. Also, the Harvard I'mUcme Analyze (HPA) 
^S^L di.tlcn.Ues UL it i******. They ^^^^"^^^^^ZX 
predicting more terminal than there were input symbols. Needless to *,y. H »p«R*k to do much belter by attach ng 
Koncr « in Farley's Algorithm IR-rtcyll* A wcH-forme* String f ^^.^^^^^ 
solve the problem, though it requires unbounded space. (It could be argued that the WFSS pn.vulcs the necessary 
bottom-nil i information by constraining the a-arch space as it does.) 
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The GP interpretations result from attaching at the critical point instead of predicting. For example, the 
machine will prefer to attach in (163), thus taking the first fatal step down the garden path. The grammatical 
(but unlikely) interpretation requires predicting a clause node instead of attaching. 

(163) [ s I told the boy the dog bit] 
[ s the dog bit] 
[ vp bit] 
= =WAIJ,= = 

W Suc] 
[ v would] 

[ v hclp] 

The "non-minimal" interpretation can be forced in the presence of positive evidence. 93 For example, (165) is 
acceptable because there is sufficient positive information (an unambiguous +en morphological feature) to 
predict a reduced relative clause 9 ^ when the machine is in stale (*66). On the" other hand, sentence (164) does 
not have Uic same reliable evidence for a reduced relative, and ben«e Aon? is insufficient motivation to 
predict the additional node. 95 Since the vp can't attach to [ np . the horse] without the reduced relative node, 
and the reduced relative node can't be predicted , the machine will ps-ctosc . the only ps-action left. In this 



93. In our formulation the positive information will be in the limited lookahcad buffer; in Fraacr's model, it will be 
discovered by ihc limited backup mechanism. 

94. The terminology, reduced relative, comes from an old deletion analysis which, derived (b) from (a) by deleting who 

was. ' 

(a) the horse who was taken past Ihc barn 

(b) the horse taken past the barn 

This construction has also been caWed wlrfc deletion (short 'for who is deletion). Instead of deleting, YAP base generates 
the construction directly as folkiws: L . the horse [ y . uikcn past the barn]]. In thfe analysis, predicting the relative clause 

amounts to rirctlicting the cp- node. 

95: If YAP looked sufficiently fartjliead. it would find sufficient evidence for the, reduced relative. We are assuming that 

one generally doesn't look thai far ahead. 
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case, closing is the first fatal step down the garden path. 96 

(164) #Thc horse raced past the barn fell. 

(165) The horse taken past the bam fell. 

( 166) [ s The horse] 

[ The horse] 
= =WAIJ,= = 
f vp taken past the barn] 
[ v fell] 

Ipunct '' 

It would be possible to parse garden paths if one looked sufficiently far ahead. Figure 7 illustrates a very 
marked rule to do so. We assume that most people do not look so far ahead because they have not seen 
enough evidence to justify the effort. I'crhaps, psychollflg^ts; wththelf unusual backgrouitd, have acquired 
a rule like the "horse-racing" rulosia figure 7.* 7 

These garden paths should be distinguished from center-embedding .because we believe no one (not even the 
best psycholinguist) can learn to parse deeply center-embedded sentences in real time. Although it would be 
possible to add a marked rule to parse garden paths, the machine is fundamentally incapable of parsing 



96. Fra/iers account differs slightly because she uses alternative phrase stmcture rules, 
np -> np vp (Frazicr) 

\ np l f np 2 the horse] [ v raced past the barn]] 

np- -> np vp- (YAP) 

fnp- fnp lhc horsc l Kp- fvp mced P^ 1 lhc barn lll 
We have attributed the problem to predicting the reduced relative node (vp-); in her framework, the problem is to predict 
the np/. The accounts are very similar (modulo the phrase structure rules). In both cases, the machine fails to predict the 
reduced relative because there is insufficient evidence. • 

97. Similarly, ii is possible to write marked production rules in YAP which violate well-known grammatical constraints 
such as Ross" Complex Noun Phrase Constraint (CNPC). Although msiwraiaLjWlfe have extreme difficulty pursing 
violations of CNPC, there arc some experienced linguists who eaiujol trust, their own intuitions became they can parse 
certain violations with relative ease. Since there are some people (e.g. experiencifti 'linguists) who can purse certain 
violations, a parser should also have this capability although it may require some very; highly marked rules. This position 
is somewhat different from [Mareus79], where it is assumed that the parser shoull be incapable of violating certain 
grammatical constraints. 
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Fig. 7. A Marked Rule to Parse a GP 

If YAP had a Rife like the ad hoc "horse-facing" rule below, it could parse, The horse raced past the barn fell. 
Of course, there is no evidence that such a rule exists, (lliis rale also has quite a number of other problems 
which will not be discussed.) 

sentence: "Hie horse raced past the barn fell." 
input pointer: 

[ 'I "he horse] 

[ np .nc horse] 

= =WALL= = 

[ raced past the barn] 

[ v fell] 

'puncfl 

(dcfrulc horse-racing 

(pattern (= s = np-) (= vp = v)) 
(action (predict \p-) (attach))) 



arbitrarily deep center-embedding. The allowable dcptli is determined by the limited memory. ° 

4.2.1 Semantic Bias 

There is some additional evidence distinguishing the GP case from the center-embedding case. Unlike the 
center-embedding case, it is possible to reverse the judgments with priming (1 §7) or strong semantic clues 
[Grain and Cokcr78] [Crain79j (168)-(173). Non-minimal attachments are generally, possible if there is 
sufficient positive evidence (linguistic training, priming, or semantic clues) to exclude the more minimal 
interpretations." 



98. It is possible to add some marked rules which would .occasionally allow an extra level of embedding. 
Correspondingly, it is possible thai a person could learn to recognise an extra level of embedding in many situations. For 
example, certain experienced psjctiolingniste have memorised certain 3-deep awstriiclions such as: #The woman the 
man the girl loved met died However, it is impossible to add enough, marked rules to allow arbitrarily deep 
center-embedding. 

99. There is one qualification: the non-minimal attachments are limited to open nodes. Hence semantic biases cannot 
influence the attachment decisions once a node iias been closed. For example, in structures like: 
[I said [i you said he said ...X..., X cannot attach lo Sj once it is closed, under any semantic context. (Some semantic 
contexts might block Sj from closing, and hence indirectly influence attachment decisions.) 
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(167) ITicrc were two horses being raced, one out in the field and the other past the barn. The horse 
raced past the barn fell. priming 

(168) Hie tenant delivered junk mail threw it in the trash. semantic bias 

(169) #'ITic postman delivered junk mail threw it in the trash. 

(170) The cheater furnished the answers passed the test 

( 171 ) #Thc genius furnished the answers passed the test 

(172) The performer sent the flowers was thrilled. 

(173) it The florist sent the flowers was thrilled. 

4.2.2 Marcus' Account 

This account differs slightly from [Marcus79], where it would be very difficult to state a rule which correctly 
resolves garden paths, and consequently, his machine will guess which path ttitatc when it carinot correctly 
resolve the ambiguity. In the garden path case, the machine will take the wrong path. 'Hie semantic priming 
can be explained in the model as reversing the heuristics. Accordingly, we would predict that (174) should be 
out since the priming has reversed the two paths. The prediction is probably correct 

(174) ?#Thcrc were two horses being raced, one out in the field and the other past the barn. The horse 
raced past the barn. 

It is more difficult for Marcus to explain why trained psycholinguists can parse garden paths. Unlike the 
priming case, the psycholinguist is aware ofboth patiis. IF the disambiguating rule cannot be stated, then how 
is it that psycholinguists seem to partseboth of then* cOrre&iy? It ^possible mat learning psycholinguistics 
increases the lookahcad buffer, and hence, they can parse certain GPs even though most normal people 
cannot. However, we have adopted another account. Instead of saying that the GP cannot be resolved by a 
marked rule, we take the much weaker position that there must be positive evidence to justify the rule. 
Marcus' position is more restrictive than our own. and hence niorc thcoitttfcafly attractive. Unfortunately, in 
YAP, it was found necessary to enlarge the class of dcfinabletuJes, andhewe, we had to abandon Marcus' 
position that the "horse-racing" rule (figure 7) cannot be stated, ni favor of the weaker position that such a 
rule is highly marked. 
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Related Work - fiJ_ 



4.2.3 Related Work 

This account is somewhat similar u>{Bcvcr7ej whcpc dicfcwas*f»»smg sttategy (175) to account for some of 
the same empirical facts. Wc have two sHght objceUorw^itfcstrategyf <a) it is not as general as Fraacr's 
.formulation, and (b) it conflates performance and competence, 

(175) Slmtfigy ]}: The first N,.V.(N),, clause (isolated by Strategy A Jwhich segments clauscsl) is the 
main clause unless die verb is marked as subordinate. 

Fra/.icr's minimal attachment also overlaps with (Chomsky and Usnik77] where some of the same 
phenomena are described in terms of filters. Fra/Jcfs account involves performance whereas filters 
presumably encode competence. Chomsky and lOTik,jftft M% (\W is ? ungrammatical; Frazicr's principle 
would imply that it is also unacceptable. 

(176) *#nic girl saw you is here. 

It is very tempting to suggest an explanation. A functionalist might argue that it is ungrammatical because it is 
unacceptable. 100 It is equally mistaken to deduce that unacceptability follows frdrti ungrtenalicality. A 
mere overlap between performance and competence docs not constitute an explanation (in cither 
direction). 101 On the other hand, the overlap is probably worth studying in more dc$J. For example, one 
might look for an explanation in terms of evolution as in {Bcvcr and l.angendocn71]. It is unlikely to be pure 
chance. 



100 A functionalist argues that a phenomenon /' is the way il is because P is a necessary byproduct erf «>mputmg tome 
function. In this case, a functionalist might conclude that minimal tfl*tM««*t«M *"»««" u«g W ro»*Ucal.ty facts 
lK-ctwse certain ill-formed sentences cannot be parsed. This position is taken in [Adcs79|. ...... tinnc .. 

101 Chomsky and lasnik specifically warn us about certain templing although hworrect^mcltonal cxpla.nal.ons. 
According u. [Chomsky and I asnik77 pp. 437]. Similar conclusions are conventional in attem a, funct.ona explanations 
MZe'Joj r physical organs, for e,un P ,e. Thus «r can no doub, account for *+!^^^*£^£ 
function ofJtpmg t**L bu, no one assumes that *■■ embryo <k* -fcto/***WP^**^f f**^ ^ ° ^ 
i^itr^Mast reasonable functional explanations are at the level of evolution. Fven ,f funcUonahsm does not 
provide an explanation, it is often useful as a motivating force. It may suggest where to concentrate the mvesl.gation. 
Although wc are not advocating an extreme functional position, it can be a profitable approach.) 
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Similarly YAP, which encodes minimal attachment, docs not explain minimal attachment or any facts which 
follow from that (e^. certain GP phenomenal but otecelyf provides a description. Wc agree with Fra/jer's 
intuition thwt minimal attachment ha conscience of linlitod ^eofees. fiven if the connection between 
minimal attachment and limited resources couW be proven, we would not have an explanation. It would 
remain to be seen why people adopt the proposed strategy in favor of some inferior one. Is minimal 
attachment learned or is it innate as Fra/Jcr suggests? iTicsclW extremely hard questions; wc have only 
attempted to model (describe) the facts. ITiis work should not be interpreted as an explanation. 

4.3 Non-Minimal Attachment 

There arc a few exceptional cases where the default order (attach, predict/and then close) would produce 
incorrect results. These exceptional cases should also be a problem for Frazicr's principles (which she solves 
with a backup mechanism.) In our framework, there will be a few marked rules to cover the following 
exceptions: 

(177) early closure (chapter 2) 

(178) transformations (chapters 6-9) 

(179) non-minimal attachment 

(180) pseudo-attachment 

Sentences (181)-(186) show that non-minimal attachment is occasionally appropriate. The first sentence in 
each group is more minimal than the others. It would appear that the parser should not blindly attach 
without looking ahead at die next constituent for one of these exceptional cases. 

(181) I know [the boy]. 

I k no w [[the boy] went home]. null complementizer 

(182) John saw Tom and [Mary]. 

John saw Tom and [[Mary] saw Sue]. conjunction 

(183) I told the boy [that]. 

I told the boy [{that] story]* 

I told tire boy {[that] you Irked the story]. lexical ambiguity 



Non-Minimal Attachment - 65 - Section 4,3 

YAP has marked rules to cover each of these cases. The last group arc disambiguated by the that-diag, a 
marked rule to distinguished the various senses of thai} 02 The first two pain art disambiguated by a marked 
rule which predicts an 5 when there is a node looking top-down for an sand'thcrc is an subject-tense pattern 
in the lower buffer. For example, an s would be predicted in (184). 

(184) [ s I] 



[ v _ knew] know-1 -> head . {obj, scomp}' 

= =WALL= = 

[ np . the boy] 

l v wcnt] 

[ n home] 

All of these examples appear to be counter-examples to Fra/jcr's minimal attachment which arc easily solved 
though a bounded kMikahcad/backup/panillcl mechamsm. There are sonie more diffki'rft examples 
(involving lexical preferences) whWh appear m supp<« drc'att^Klcriiig hyfMimdte. Se^ 
a typical minimal pair illustrating the difference between see and know, which cannot be distinguished in 
purely structural terms. Although we have not implemented a solution, we sec no reason why these facts 
favor backup over lookahcad (or parallelism). 103 

( 185) I saw f s the horse raced past the barn). 

(186) I knew [ the horse raced past the barn]. 

4.4 Pseudo-attachment 

There arc structurally ambiguous scntcrces, vwlating any well ordered set of principles; these should be 
recognized as ambiguous (or perhaps, vague). These present a problcift<ft)r both Fra^ei^s principles and our 
ordering heuristic. YAP detects the ambiguity with a marked rule. Frazicr'siwu principles seem to conflict in 
this case, in the sentences below, minimal attachment would attach $c, #/* high and late closure would seem 
to attach it low. , ,. 



102. Martin (personal communication) has infonaud us that certain senses of that were nipre uniform in older forms of 
Fnglish. It is quite possible that we are missing a gcnenili - it ion in the various lexical fojms of that, 

103. In a parallel model, one amid imagine that unusual lexical entries, wouklUtkc, longer to fetch from memory, and 
hence, an unusual sense wg^Jd lose the "race". In a lookaheud sysionv it is pcjssiblelo stale the marked rules so they will 
trigger very rarely (only in the marked case). 
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( 187) Sam hit the girt with a book. 

(1^) Sam hit [the girt] wim a book, high attachment 

( 189) Sam hk ; [thc girl with a book], low attachment 

llicrc arc several possible ways to deal with this apparent conflict. 

(190) Define one oftlic two principles to avoid the problem. (Frazicr's solution) 104 

(191) Cope with the possibility of conflict. (the "double finish" accojunt) 105 

( 1 92) Add an additional rule to cover the conflicting cases. (YAP's approach) 

YAP has a marked rule to pscudo-attach (attach both ways) 106 when it sees both alternatives and decides that 
it cannot decide which is correct, 'lliis approach is completely consistent with Marcus' determinism 
hypothesis, YAP makes a single left to right pass over the input stream without backup. Once it 
pseudo-attaches, it will not retract the decision at n later date. In this way, Marcus' determinism hypothesis 
allows ambiguity, even mough a deterministic PIM cxcltKtes ambiguity. 'Ilic following sentences illustrate 
pseudo-attachment: 107 

(193) Put the block in the box on the table. 

(194) He carried nothing to indicate that he was one of the group. 

(195) We sighted the man with the binoculars. 

(196) Wc never fought a bull with real courage. 

(197) He hit the man with the stick. 

(198) He seemed nice to her. 

The cstnicturc representation of these sentences in not a tree but rather a directed aevclic graph (DAG). 108 
For example, L to her] in 098) would have iwo, mothers: Ac participial phrase f v seemed ...J and the 
adjectival phrase{ ap _ nice ..,}. The multiple mothers should be interpreted as exewsive possibilities, lliis is a 



104. Frazier [Frazicr79 pp.14.1) argues thai her late closure principle docs not apply here because the girl with a book is a 
single package. As she defines late closure, il works on packages which are roughly six words long. 

105. Suppose (hat the parser consisted of several parallel processes which were all competing against each other. The 
first process to finish would he the "winner" and its output would be UikeirHS-thv preferred interpretation. When two 
processes finish at the same time, the sentence might be considered ambiguous/vague. 

106. This idea was first suggested by Witch Marcus. It is' 'similar to Sageriintf Grishman'S notion of permanent 
predictable ambiguities. [Grishman73] ' ■ -■ ■■■m ■ ■ 

107. Many of these sentences are from fWales and Toncr76j. 

108. A DAG is a general graph (of nodes and relations) with a condition excluding circular loops'. Alternatively, a DAG 
is a generalization of tree where daughters may have multiple mothers. 
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convenient way to represent certain common structural ambiguities that occur in natural language. The 

: ^; v.. ■■■■■. -■■■'■■ - .., "•■ •■ 

cstmcturc of (198) would have the following representation: 

(199) [ s He [ vp seemed [ ap . nice PPj] PPj]l 
where PPj = L to her] 

ITierc arc three interesting cases of pseudo-attachment illustrated by (20O)-(202). In all three cases, downl 
can attach to cither upl or up2. (Sec figure 8.) In (200), up2 optionally selects another daughter, whereas in 
(201) and (202), up2 obligatorily requires another daughter. In (201) unlike (202), there is another 
constituent, so pseudo-attachment is possible. Ilwrc is a marked rule which considers the three possibilities. 

(200) lie [ up2 seems [ upl nice | downl to her ... pseudo-attach 

(201 ) [ u 2 Put [ up j the block [ down y in the box on the table ... pseudo-attach 

(202) [ 2 Put | ! the block [ downl in the box. don't pseudo-attach 

Pseudo-attachment is not limited to just prepositional phrases; the YAP mfipTcmcntation generalizes the 
technique to work for any kind of xp- (pp-. ap-, or vp-), not just pp-. Consider die following examples: 



Fig. 8. Pseudo-attachment 

sentence: He seems nice to her. 
input pointer: 

L he seems nice] 

[ v _ seems nice) 

l a p-niccj 

= =WALL= = 

[pp. to her] 

[pptohcr] 

Ipunct '' 

The marked pseudo-attachment rule attaches downl to both upl and up2. YAP knows that it cannot 
disambiguate between the twapossibilities. 



Pseudo-attachment -68- Section 4.4 



(203) Put the block [ in the box on the table. 

(204) I considered every candidate [ likely to win. 

(205) He carried nothing [ to indicate that he was one of the group, 

YAP uses a very similar technique to process certain well-known cases of ambiguous wh-movement such as 
(206). These will be discussed when we consider wh-movement in chapter 8. (206) has two interpretations: 
(207) and (208). Uoth of these arc represented wittun a single structure (209) where the trace NP-j has two 
mothers. 

(206) Who(m) do you want to sec? 

(207) Who(m)j do you want to sec /j? 

(208) Who(m)j do you want i- x to see? 

(209) Who do you want NP-j to see NP-j? 
whcrcNP- i = [ np J 

The pseudo-attachment technique follows a ppjpular philosophy in artificial intelligence called dslajJSd 
binding . JTic basic idea is to avoid making ajbitraxy. decisions until, mc^rc is enough information. This 
approach can be contrasted with an arc ordering technique (such as [Kaplan72]). In Kaplan's scheme, the 
possible decisions arc ordered so the most plausible decisions arc made first. In a delayed binding scheme, 
the system tries to avoid discriminating between possibilities as long as possible. In some cases, the system 
may never really distinguish between certain possibilities. 110 



109. Wh-movement refers to a class of constructions including relative clauses and wh-queslions. These constructions 
relate a wh-word with a gap which is represented by a t (for trace). Traces are represented in YAP as phrases which 
dominate no words. 

relative clause: I saw a boy who. you know L. 
wh-qweslion: Who- t did you sec tj? 

This will be discussed in "more detail* in chapter 8. •*-..- 

1 10. [Vanlxhn7X] observed that informants sometimes claim they understand a sentence with multiple quantifiers until 
they are asked questions regarding quantifier scope. The subjects will often admit they hadn't considered the scope issue. 
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There arc limitations to the particular implementation of delayed binding in YAP. It may be impossible to 
encode all grammatical ambiguous interpretations. Wc claim ihat^i^lo-aftachmentcan work for acceptable 
interpretations; the other grammatical interpretations arc unacceptable. Unfortunately, it is very hard to test 
this claim. 

It appears that pseudo-attachment cannot represent all CF interpretations because the device docs not have 
CF generative capacity. One could view pscudo-atlachmcni as annotating one of the attachments (the 
canonical attachment) with several alternatives. The weak generative capacity will be the same as the 
canonical structure; pseudo-attachment docs not affect the weak generative capacity, only the strong 
capacity. ' ' ' Assuming that YAP is equivalent to a deterministic PDA, 112 it has the Weak generative capacity 
of a deterministic language (i.e. I.R(k)). Since l.R(k) languages do not include all CF languages, there arc 
some CF languages which cannot be described using pscudoraitacJimcnt. ' J Wc claim that acceptable 
sentences can be described with pseudo-attachment. 114 

Pseudo-attachments should not be undone at a later date. There arc certain problematic cases where the 
simple scheme described above will run into trouble. There are several possible replies. Some of the 
interpretations arc probably unacceptable. Perhaps the rest could be processed with more lookahcad. There 
are some problems with pseudo-attachment; nevertheless it is an interesting alternative to purely 
non-dctcrministic strategies. 



111. The weak generative capacity is Ihe set of sentences generated by a particular grammar. The strong capacity is the 
set of derivations . In general, ihe strong capacity is much larger since an ambiguous sentence corresponds to several 
elements in (he strong generative capacity, but only one in the weak generative capacity. (Since the class of the machine 
(FS, CF. CS. TM) is lied to the weak generative capacity, pseudo-attachment can be implemented without moving to a 
higher computational class.) 

112. It is conjectured that YAP would be a deterministic PDA if the stack bound were removed. 

1 1 3. For example, there is no I R(k) grammar for an inherently ambiguous language. 

114. This assumes thai acceptable sentences form an I.R(k) language. Even stronger, this result should follow from 
Marcus" Determinism Hypothesis and not from our FS hypothesis. (It trivially follows from the FS hypothesis since all 
FS languages are also I R(k).) Otherwise, it isn't clear how ambiguous parses amid be found short of exploding the stale 
space as suggested in chapter 1 when Marcus' Hypothesis was first mentioned. ' Tn other words, we arc assuming that 
acceptable sentences (even wilh arbitrary center-embedding) are still weakly equivalent to an I R(k) language. 
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(210) Put the block in the box on the table PP* into the basket . 

(21 1) 1 consider every candidate likely to seem XP-* corrupt 115 

Sentences (210)-(211) illustrate a problem with pseudo-attachment; the final constituent, which is arbitrarily 
far from the decision point, selects the higher attachment as in (212H213). But without the final underlined 
constituent, the examples are highly ambiguous as (214H21S)ilWstratc. The problem is that YAP has to look 
at tlie final constituent before it can determine whether or nt*to pteiidtf-ftttacH: (Tic final constituent might 
Ik arbitrarily far away. 

(212) I>ut [the block in the box on the tabic PP*] [into the basket], unambiguous 

(213) I consider (every candidate likely to seem XP*] corrupt 

(214) Put [the block] {in the box on the table PP*]- highly ambiguous 
Put [the block in the box on the table] PP*. 

(215) 1 consider [every candidate] [likely to seem XP*]. 
I consider [every candidate lifcely to seem XP*}. 

Wc will make a simplifying assumption that the intermediate phrases all attach the same ways. Only the first 
and last few phrases in a sequence (XP*) can be pseudo-attached; Jt t js assumed that die intermediate phrases 
all attach the same way. Consequently, the pseudo-attachment decision depends on just a few phrases (downl 
and down3 of (216)), not on an unbounded number. 

(216) [ s put the block] 

[ put the block] 
[ np _ the block] 
= =WALL= = 

[ pp . in the box] the first xp- 

pp* 

,r the middle xp-* 

[pp. into the basket] thetastxp- 



1 15. This example was suggested by Joan Bresnan. 
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Certainly there arc numerous grammatical interpretations which cannot be dpj»ribcd by th is mechanism. For 
example, there arc an unbounded number of grammatical interpretations; this mechanism only considers a 
bounded number: 116 

(217)lput[thcblockpp*]pp 
(218)1 put [the btock ppj] pp* 

(219) I put [the block] pp* 

Wc claim that the others arc unacceptable (in die absence of positive evidence such as semantic bias). ITicrc 
could be marked rules to consider semantic or pragmatic clues. 

4.S Summary 

The cstructurc implementation has been outlined. Unless there is an applicable marked rule, the interpreter 
runs the phrase structure rules in an unmarked order, The unmarked order was chosen to be compatible with 
Fra/icr's two principles: late closure and minimal attachment. Wc have dasttusscd several classes of marked 
exceptions (220)-(223). 'I"hc description would be more attractive if die role of tiicsc marked exceptions could 
be minimized. This is an area for future research. 

(220) early closure (the A-ovcr- A closttPe principle) 

(221) transformations 

(222) non-minimal attachment 

(223) pseudo-attachment 

In the next chapter we will show how fstructurc can be built from cstructurc without violating memory and 
backup limitations. 



116. There would be one other interpretation if put didn't suhcalcgoruc for an obligatory second object: 
/ stiw [the block pp*J. 
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5. Functional Structure Implementation 

The previous chapter sketched out YAP's basic machinery for constructing the constituent structure 
(cstructurc), based solely upon category (n, v, np, vp.s, ...) information. The cstructurc is an intermediate 
representation toward obtaining the predicate/argument relations ((structure).. Computing the fstructure 
involves a number of syntactic features (properties). It is easy to find minimal pairs such as (224H229) 
illustrating the necessity of certain syntactic features. 

(224) That ba]J is round. number 

(225) That balls are round is a fact. 

(226) Have we. eaten? case 

(227) Have ys eaten! 

(228) Have the boys lake, the exam! tense 

(229) Have the bovs taken the exam? 

Kach node (phrase) has a number of syntactic features (eg. person, number, gender, case, tense and mood) 
and a number of grammatical roles (eg. subject, object, etc.) ITiis chapter will outline a procedure for 
assigning features and roles. The problem is interesting because feature dependencies can cross seemingly 
unbounded distances. Nevertheless YAP has a procedure for manipulating features that doesn't violate the 
severe resource limitations (memory and backup). The feature manipulation problem is similar the 
inheritance problem [Fahlman77] [Martin79, 80], which is known to be very hard. Fortunately, the 
Brcsnan- Kaplan linguistic theory provides us wUh just the necessary simplifying constraints. 

Many parsers compile the feature information into the parts of speech (category), conflating constituent 
information (n, v, ...) with functional information. Perhaps the most extreme example is the Harvard 
Predictive Analyzer (HPA) [Kuno66] which used about 180 parts of speech to distinguish everything from 
number to subcatcgorization frames. We accept the proposal that the two structures should be 
independent. In addition to her linguistic motivations, there are some computational advantages for 
dividing the problem in this way. It is often useful to delay certain decisions as long as possible. The HPA, 



117. The independence property is central to the Brcsnan -Kaplan framework though it has appeared in earlier models 
including ATNs. 
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with its 180 parts of speech, couldn't separate the distinctions which require immediate resolution from the 
ones that should be delayed. Consequently, it found many more ambiguities than most people consider. For 
example, the HPA finds three interpretations of (230) wlwreinost people notice only two. if that many. Some 
of these distinctions should be delayed (perhaps indefinitely). The multiple mtcipWtatidns of flying planes 
arc far more striking than the possible senses of am 

(230) They arc flying planes. 

(231) lliey arc aux [ vp flying planes] 

(232) Hiey arc copu|a [ np flying planes] 

(233) They are copuU| I vp flying planes] 

YAP, as opposed to HPA, carries along multiple functional possibilities until tlicre is some reliable 
information to resolve the various alternatives. In this way, YAP can manipulate feature dependencies over 
unbounded distances without violating Marcus' IX'tcrminism hypothesis. 

5.1 Seemingly Unbounded Dependencies 

We will illustrate a typical "unbounded" dependency m the features between two nodes and then snow how 
the dependency can be captured with only finite memory. The method is in fact fairly general since it is based 
on the Brcsnan-Kaplan linguistic theory. 

(234) There is a problem. 

(235) There arc problems. 

(236) There are a problem. 

(237) *Thcrc is problems. 

There-insertion sentences such as (234)-(237) have two dependencies: 

(238) subject-verb 118 agreement 

(239) there agrees with its object 



IIS. Qranimalical roles (subject, object* predicate, etc) will be undefined for the time being. The intuitive notions 
should suffice for the current discussion. 
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'These dependencies can cross an unbounded amount of material as me following sentences illustrate: 

(240) There seems likely to seem likely to seem likely _. to be a problem, 

(241) There seem likely to seem likely to seem likely „,. to be problems, 

(242) * ITierc seem likely to seem likely to seem likely ... to be a problem. 

(243) * There seems likely to seem likely to seem likely ... to be problems. 

In these raising 119 sentences, each embedded phrase takes an understood subject. The dependencies can now 
be suited locally, although they have unbounded consequences. 'ITiat is, flic highest subject agrees with the 
tensed verb and the most deeply embedded subject agrees with the object Furthermore, all the understood 
subjects arc related, so they inherit each other's constraints. Much of tliis chapter is concerned with the 
inheritance mechanism. 

(244) Thcrc2 seems x^ likely x^ to seem ..i» n to be a problem. . 

Wc will use a variable x as a place marker to represent the understood subjects,. Now die two dependencies 
arc local; ihere-y agrees with seems and x agrees with a problem. Since the subjects arc related, the procedure 
has unbounded consequences. Nevertheless the procc&ire docs not require inordinate resources. 

5.1.1 Grammatical Roles 

The notion of subject is crucial to Uiis formulation. The Rrcsnan-Kaplan analyses use a number, -of 
grammatical roles including subje ct, object. obj2 (second object), xcomp (adjectival, verbal, or prepositional 
complement), scomp (sentential complement) and pred icate. Grammatical roles arc assigned by structural 
and lexical constraints. For now, wc will give an example to illustrate the intuitive notions: 

(245) subj I saw a boy. 

(246) obj I saw a bov . 

(247) obj2 1 gave a boy a kail 

(248) xcomp He seemed likely to. bjj nice . 

He seemed to be nice. 
I gave a ball to, a bov . 



119. Raising is a particular linguistic construction which has received considerable attention in the linguistic literature 
(sec [Pusial74] for a long list of references). 
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(249)scomp It seemed that he was pice . 

(250) prcd I saw. it. 120 

These arc all slots in the fstructurc. Grammatical relations arc extremely useful for describing many linguistic 
phenomena (sec (BrcsnanMH). 121 

5.2 Constraint Propagation Solution 

ITiis feature manipulation procedure can be viewed as a constraint propagation problem [McAllcstcr80] 
[Mackwoi ih77J [Walt/75]. The problem is to propagate the agreement dependencies ill rough the fstructurc (a 
graph of grammatical roles). (Sec figure 9). Initially, all possible values arc assigned; the number values are 
{singular, plural}. 122 Extraneous values arc first weeded away by the lexicon and *cn by agreement 
constraints. In this way, multiple possibilities are carried along until there is sufficient information to 
disambiguate. YAP docs not randomly try alternatives (non-deterministic); heuristic guessing is avoided 
whenever possible. 

Figure 9 shows an fstructurc after lexical specifications but before the constraint propagation. Kor example, 
the lexicon specifics that a problem is singular ({singular}) and there is cither singular or plural ({singular, 
plural}). After propagating the two agreement constraints, ^ x 4 and x 6 wi " a ^ ** sm 8 u,ar ( thcir IMQbcr 
properties will be {singular}). 'ITic sentence. There seemHfkttyto be probtentx has a similar fstructurc except 
x 2' x 4' x 6 iin< * x 7 arc P' ura ' ' nstca d of singular. 



120. In the llresnan-Kaplan framework, pred is a feature, not a grammatical ix>lc. Wc have placed il here because it is 
defined over a large set unlike the other features such as pcrsolt:'nWmb*»r and gender. 

121. Chomsky (personal communication) has criticized grammatical relations as an inadequate explanatory theory. 
Although it is possible lodcscribc dw facts starling fto»» graHmwtknJ- fetation*. » truly explanatory theory would have to 
derive grammatical relations themselves. Chomsky argues that deriving grammatical relations from structural notions is 
the hardest part aiwlconsMii^ulyvJhe'hulKwfMH very tffcfui #a ti^h^e theory: THiis point is extremely controversial. 
Nevertheless, explanatory adequacy is somewhat orthogonal lo processing issues; for our purposes "mere" descriptive 
adequacy is sufficient. (Descriptive adequacy is no simple task.) 

122. We are assuming that features are defined over small sets of possible values. There arc some theoretical difficulties 
associated with propagating grammatical roles since they have potentially unbounded ranges. The actual implementation 
has a special symbol {^undefined*) for the universal set of grammatical roles. 
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Fig. 9. Const mint Propagation 

There seems likely to be a problem. 

Thcrc 2 sccmsj x 4 Iikcly 3 x 6 to bc 5 a problem^ 

The fstructure graph (before propagating the agreement constraints) is given bchJi* (omitting certain details). 
Hie two agreement constraints arc subject-verb agreement (x 2 with Xy) and there-insertion (x 6 with x ? ). ITic 
constraints arc sufficient to uniquely determine the number features ({singMjajrjJ. 

before after 

Xj prcd: seems 

tns: {pres} 
subj:x 2 

xcomp:xj 

x 2 form: there ;i 

num: {singular, plural} {singular} 

X3 prcd: likely 

sub J :x 4 
xcomp: xc 

x^ is-bound-to: x 2 

num: {singular, plural} {singular} 

xj prcd: there-be 

subj: x 6 
obj: Xt 

Xg is-bound-to: x^ 

num: {singular, plural} {singular} 

X7 prcd: a-problcm 

num: {singular} {singular} 



The two constraints arc subject-verb agreement and 0>erc-inscrtian, ln n M>is framework, subject-verb 
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agreement is enforced by intersecting the agreement features of a ]SBSSA node with its subject. 1 In figure 9, 
the number features of the tense node Xj arc intersected with its subject x 2 , making x 2 's number {singular}. 
Similarly, tlwrc-insertion constrains x 6 to agree with Xj, making * 6 's nurftber fcaftirc {singular}. By 
is-bound- to edges, the agreement constraints pjopapte all theway tbAHHjr* the graph, making all the number 
features {singular}. 

If the constraints were inconsistent , some slot would have no possible values, and the sentence should be 
ruled out. For example, the ungrammatical sentences, *There seem likely tp be a problem and 'There seems 
likely to be problems, arc bad because their fstructurcs have no possible values (i.e.J}) for the number slots; 
one agreement constraint weeds out the value singular and the other removes plural . Hie ungrammatical 
sentences arc functionally, inconsistent 

I f the constraints imdcrdctcrminc the solution, some slots will have several possible values, and the sentence is 
considered vague (or perhaps ambiguous). 124 The number features in (251) and (252) arc all {singular, 
plural} indicating a number ambiguity. In (251) there may be one or more "deer"; in (252), there is an 
ambiguity between the inner and the outer interpretation. 125 Sentence (253) has undcrdctcrmincd tense 
({pros, past}) since put is lexically ambiguous. The undcrdctcrmincd cases illustrate that the cvaluatorcan be 
so lazy it may never get around to making a decision. 

(251) The deer might be nice. 

(252) The family might be nice. 

(253) I pjU it down. 



123. Actually, tensed verbs don't have number features themselves, but miner assign number features to their subjects. 
For example, seems assigns singular features to its subject, although it is not singular itself. This point is important in 
examples like That they seem to be tike Is afoot where the einbcdded dause ; ftitlNguttr l «Vi-n lhw<gh its inain vcrb^mw) 
assigns plural features to its subject {theft. ir •■••.-•- i* r 

124. An ATN modcf can distinguish betWWft M^^^ ;md «/»^i«^ 1^ 

values (vague) and non-delerministie assignments (ambi}*t»Wy). fct Otir frstmcwwrfc, wH dftnt have the second rrtcchahism 
and hence we cannot (currently) tfisttHgtrish the two cases.' 

125. Collections can be viewed as hianfy^rtdKidiial entities' (initer), and hence plural or they can be viewed as a angle 
conglomerate (outer), and hence singtrtiW. -,>■>■•■. 
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5.2.1 Representation Issues 

Feature values arc represented in bit vectors 126 so that each set (t.e.,{singular, plural)) requires a constant 
amount of memory (independent ofits size.) That is, theset {Snigtifar} «rtd the set {singular, plurar} require 
the same amount of memory. Unlike most non-deterministic systems, the ambiguity does not consume 
additional resources (time or space); the number feature requires exactly one bit vector in any case. Hicsc 
representation issues can have a fairly important impact on the overall performance of die system; it is often 
worthwhile to take advantage of the particular parallel construction of the machine at hand in order to avoid 
potentially expensive non-deterministic searching. 

The features in figure 10 have been implemented. 127 Kach possible value is represented by a single bit; 
1 = possible, = impossible. For example, if the gen and dat bits are set, then the case is cither genitive or 
dative ({gen, dat}). In this representation, it is particularly easy to merge nodes; we simply intersect the two 



Fig. 10. Features 




feature 


possible values 


case 


gen dat nom ace 


gender 


mnf 


pne 


si s2 s3 pi p2 p3 


def 


+ - 


pro 


+ - 


tns 


tnslcss prcs past -f-ing +en 


mood 


deel wh-q yes-no-q imperative exclamation subjunctive 



126. A bit vector is an array of binary variables. It is very similar to standard set of binary valued features. We have 
chosen iliis representation for efficiency reasons: il requires minimum space and certain operations (store, fetch and 
merge) can lie done in parallel because I ISP h;iS(jpcralions for perfijrming, lexical operations in parallel on a single 
machine word (32 or 36 bits depending on die particular hardware). •.■;,.■„, 

127. Category (s. n, v. ...) is not implemented in this way because aaqgocy, f€«A»fcs are not percolated through the 
fsirucUire like the others. For example, although there is good, evjjdence thill a,;ttOu« phrase inherits a number value from 
iLs determiner (this boy, these boys), it is much harder to argue Dial il inherits a category value. Category is defined to be 
part of the cstruclure. 
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bit vectors. 128 Wc arc crucially depending on the fact that features range ovcra small finite set of possibilities. 

5.2.2 No Disjunctive Constraints 

There is a crucial linguistic assumption that enables the constraint propagation technique to work: there arc 
no disjunctive constraints. It would not be possible to enforce a rule for example that required the first 
daughter to agree with cither the second qt third daughter. Disjunctive dependencies arc known to be 
computationally difficult because they involve postulating several possible worlds which may have to be 
considered non-dctcmiinistically; fortunately they don't often appear in natural language syntax. 



128. Person and number have fecert combined (pne = person/number aide) because" 'there are often disjunctive 
constraints between the two. For example, the noun block can. lake an) pctstit) ; value and any number value, but the 
values are not 'independent (it cannot be s3 - third person singular). This encoding trick is taken from Parsifal. Kaplan 
(personal communication) mentioned that his A- FN parser used the same Irkk. ^One could argue that tns and pne arc 
somewhat analogous: there are some words which have either tux features or pne features, but not both. For example, the 
lexically ambiguous word blocks is either pres or s3, but not both. Tins idea has not been implemented.) 

12V; Martin (personal communication) itnrjws of only on* ^ytHaetic tfiiisiriicliori which suggests disjunctive 
dependencies. The partitive noun pbnsc kind oj dugs mjgjit be either singular or pJural. M seems to inherit its features 
from one i if the other of its parts (but not necessarily both). 

What kind of dogs are those? 

What kind of dogs is the most popular? 

Perhaps kind is not {singular}, but rather it is vague ({singular, plural}) between the inner and outer plural. The 
following pairs illustrate similar ambiguities. 

The bellows are coming apart. 
The bellows is being repaired. 

The committee are fighting among themselves. 
The committee is fighting the regulation. 

This approach avoids disjunctive constraints, which are computationally problematic. Instead of postulating an arbitrary 
number of possible worlds, there is only one possible world which encodes the ambiguity (i.e. {singular, plural}). The 
system will not hypothesize which possibility is correct until there is sufficient information to be sure. In truly ambiguous 
sentences, the distinction will never be made. 

The deer might have done it. 
The fish shouldn't have. 

This is consistent with the wait and see approach. (The set of possibilities (i.e. {singular, plural}) are stored in bit vectors; 
the information associated with a set it independent Of the number of possibilities.) 

Kaplan (personal communication) has suggested thai lexical ambiguity and lexical redundancy rules are a very serious 
source of disjunctive constraints. His point is well lakcu. though progress is begin made. Robert Milne is currently 
working on the lexical ambiguity problem [Milne78a, 78b, 79, 80]. We will discuss our own solution to optional 
transformations (and lexical redundancy) in chapters 6-7. 
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5.2 3 Bind* is an F^valcacc Relation 

There is another useful simplifying assumption: the is-bound-lo relation** Ibrms natural equivalence 
classes. 131 We will replace the relation with its reflexive, symmetric, transitive closure: bind *. In figure 10, 
the embedded subjects arc all bound to one another forming a single equivalence class (under bind*). 
Kquivalcncc classes can be represented very efficiently; instead of storing each clement individually , it is 
possible to store them collectively as a class, often saving considerable memory. The equivalence relation 
representation contains far less in formation "than an arbitrary relation. 'Phis is very important for YAP, since 
there may be a bounded number of classes, even though there arc an unbounded number of elements. 

The equivalence property is a stipulation. We cannot currently cxptam^hy ttfits theempirical data as wctt as 
it docs. The theory wp^ld^bc more attractive if this assumption did »ot haw to be stipulated.- 32 It may be 
possible td explain it in terms of other Ind^cri'dcntly moii Va^l; ^ssu^r4p1(lons. fapvcrthclcss, it seems to be 
consistent with the facts and it enables a computational optimiatfioifc!? 5 < ■ 

YAP does not assign features U) nodes individually, but rather to ^vatencc classes colle ctively. All the 
coHnctexcd subjects in figure 10 wonld share a single bagttftefcutas.* 3 * TOit ! iS;fe, xiartdf xi in figure 10 are 
represented collectively in the optimized (structure (256) under x 2 . In many parsers (including Marcus' 
Parsifal), each embedded subject would be represented individually. ., f i( 



130. Our use of is-bound- to is very similar lo transformational movements in Cbdm#ky"s framework. When mbmd two 
positions, he would move a constituent from one position lo the other. -■..:>•■■!■ 

131. This properly is implicitly assumed in {Kaplan and Bresnan80J. ...... 

132. Il^c^re some very ^ 
equivalence relation: frwmsk/H^ 
evidence to decide the irtaticr.(ito^ 

is an additional stipulation and hence il is undesirable. But on the other hand." trtc equivalence relation requires less 
in formation to represent (than a more general relation) and hence il is lo be preferred. There-cis a. certain advantage in 
having a more restrictive theory. It is not clear whether it is theoretically more desirable to haw fewer stipulations or a 
more restrictive representation. 

13.?. Although a processing argument alone' is not udc^tiatc Juslihci|ik>n fdr adopting the proposed ;issuntption 
(movcmeiu is equivalent u> hind*), it srwuldi)^ 

134; Clnintsky (personal aiinmunication) hai proposed that case might feigned 'to each index (i.e. each equivalence 
class), hot iinndividual noun phra^ 

13$.' This is inefficient in both space- attd time. In Parsifal, for example, it tan take unbounded lime to trace the binding 
pointers back to the lexical subject > T 
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(254) There seems likely to be a problem. 

(255) Iticrc^ sccmsj x 4 likely} x 6 to be 5 a proMcm 7 . 

(256) xj prcd: seems 

tns: {pros} 
subj: %2 
xcomp: Xj 

x 2' x 4' x 6 '" orm: t * 1cre 

num: {singular, plural} 

Xi prcd: likely 

subj: x 4 
xcomp: xj 

xr prcd: there-be 

subj: Xg 
obj: \-j 

\j prcd: a-pr©blcm 

num: {singular} 

Co-indexing is a unification procedure. Whenever two nodes arc co-indcxed, their features are merged 
(intersected) and placed in shared memory. Updating one node's features would affect the other because 
their features arc being shared. In this way, an unbounded number of nod^^^uld be affected with a single 
update, since they might all be sharing the same features. This is how the "unbounded" dependency in (254) 
can be realized with only limited working memory. 

Although the dependency is "unbounded" in cstructure. it is bounded in fstructurc, which uses the more 
efficient equivalence class representation. A grammatical role (i.e. subject) refers to an entire class (with 
potentially unbounded membership) such as (x 2 , x 4 , x 6 }, not to an individual member. Consequently, it is 
possible for YAP to enforce these agreement constraints very efficiently in die fstructurc since they mention 
only a bounded number of classes (grammatical roles). 136 



136. In the Branun- Kaplan framework, agreement dependencies are not allowed to reference more than four 
grammatical rules in a single rule. 
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Another attractive computational property of equivalence relations is, asfiociau vitv : thev can be constructed in 
any order. x 2 could be unified with x 4 and then with x 6 or the other way around. Tlfcfcmicture witt turn out 
Uic Siimc whether constraints arc propagated cyclically 137 (bottom to top), inverse cyclically (top to bottom), 
or inside out. The results are invariant with the order of application. In variance is very comenieiu; a parser 
can then enforce constraints in the most natural order (left to right). 

Invariancc docs not follow from most definitions of movement because a lexical object cannot be moved until 
it has reached the source of the movement Consequently it mak9> a difference whether movements arc 
computed cyclically or not. Perhaps movement should be redefined to be associative. 138 Similarly, die A IN 
Si:NI)R operation (which manipulates feature registers) is non-associative, lilts too could be redefined. 
Actually, part of the motivation for defining the Brcsnan- Kaplan merge operator wa£to rid the asymmetry of 
the A IN SKNDR [Kaplan (personal communication)]. 

5.3 The Brcsnan-Kaplan Analysis of There-insertion 

We will compare our analysis with the Brcsnan-Kaplan analysis; YAP wasFduagncd so th«U it could easily 
incorporate many of their ideas. Conscqucndy, we were able to borrd«f Smany analyses, such as the 
formulation of there-insertion, saving us considerable time and energy. We arc not interested in reinventing 
all of linguistics; this thesis is mainly concerned with processing constraints. 

■ , ■ . ■'..,■,. ,!■..-..■,.'? ...-...,'■ 

The problem is to build a fctructUre ft&m the cstrtittufe' ThccoHstraltrts'on the Structure come fibtn the 
cBtructurc (eg. the subject is the first rtp undeftertse) and the Ml!»n. ASt^plcnl stWtttnWdcpdntichcy rtiateV 
a noun phrase in "subject position" (immediately dominated fcyW^s^cfeifc^fth^ 
Similarly, there arc lexical constraints indicating, for example, that problem is {singular}, problems is {plural} 
and deer is {singular, plural}. (2$7M2$8) link (structure posidonrwith gnimmatical rolcs^ the remaining 
functional slots will be filled in by the lexicon. 



137. [Frciclin78J has observed that cyclicily is derivable from independently motivated assumptions. In this framework, 
the cyclic order generates the same results its any other order. We could interpret Freidcns results to say that order is 
irrelevant: the facts that were once explained using ordering constraints arc covered under more general binding 
conditions. 

138. A movement could for example leave a sink behind to swallow up the lexical phrase when it finally docs arrive 
There could be a well- formedness condition, blocking final stfwterMS,«»t«UBig*»j^ 

indexing scheme [Koster78].) ; , 
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5.3.1 Structural Constraints 

(257)up:s->dl:npd2:vp 
dl = subj(up) 
d2=±ui* 

(258)up:vp->dl:v(d2;np)(d3:xp-) 139 
dl = up 
d2 = obj(up) 
d3 = xcomp(up) 

Kxamplcs (257)-(258) arc a slightly modified form of Brcsnan-Kaplan's notation. It has been changed to more 
closely resemble YAP's notation and to be easier to type. 140 Roth (257) and (258) show a phrase structure rule 
followed by a number of constraint equations . For example, (257) gives an expansion for r, it has two- 
daughters, the first is an np and the second is a vp. There arc two constraint equations below the ps rule which 
fill in functional slots by a unification (co-index) operation. For example, the first equation, dl = sub/up), 
defines the np under 5 to be the subject, 141 by unifying the first daughter (an np) with the swfy' slot of up (an s). 
After the two nodes have been unified, they share the same memory so that further constraints on cither node 
will affect die other. Hence the unification operator is the bind* cquivatance rcUrtiom the glasses are 
represented collectively in shared memory. 

The second constraint equation dl = up unifies the head of a phrase with its mother. ITiis follows from x-bar 
theory [Jackcndoff77] [Chomsky70] where phrases arc defined as a projection of a head. For example, a noun 
phrase, such as the /Ae/wj^is a project^ of its h^ ¥• 

Again, from x-bar theory, it follows U*n *M feattire^ pcrcftjat^up Aom 4he. head. For example, the noun 
phrase the boy is singular because its head ■& singular. Sfyntlarift Jjl^awhipijjbas past: tense because its head 
vp has past tense. Functionally, one cannot distinguish a mother from its head, and consequently, they are 



139. The pseudo-ealegory xp- slands for ope of the following: ap-. vp- orpp-. 

140. YAP uses more mnemonic names: names tikcdl. dl. ... dn arc replaced with csubj, cobj, excomp. .... The letter c 
indicates a cstruclural relation, as imposed lo an /for a functional 1 role. #hci* We have used up and d. Bresiian and 
Kiifilan woutd use tip Arrows : anft&iwn arro^fc respettiv^rj'. Afco: ilfflc^tif tiiinibcffn^^iaughtcrs as we have, she 
writes the constraint equations underneath the approprfttte (laughter. CeWaWtxMrriiift ekjtiattons can be tmfierstood as 
the unmarked case. so Ihey need not be resulted for each ps rule. See p(apfen ! Wtd f Br«Hkiti^; ' u 

141 Technically, the subject is (he jfsltucfare of die np under iHtot iMvipWtff" fh* subject docs not mditdc the 
c structure of the np (category Ind surface chnrghtefsj. 
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represented as a single unified node in fstructure. 

5.3.2 lexical Constraints 

llic remaining constraints come from die lexicon. A lexical entry looks very similar to a phrase structure rule. 
It defines a functional frame (instead of a constituent frame) with constraint equations between slots. We 

have used the dummy variables al an instead of dl dn to distinguish ftrt*£tkHral arguments from 

constituent daughters. ITic following lexical entries arc relevant to the example at hand: ,42 

(259)ssejn->al:{vp-,ap-} 143 
al = xcomp(up) 
subj(up) = subKal) 

(2<i0)likjjli->al:vp- . 

al = xcotnp(up) I 
subj(up) = subj(al) , , ; 

(261)Ujcj£±£->ar:np- 

al == obftup^ "■■-'■ ' m; 

num(subftup)) == num(al) 

form(subj(up)) = there ; ;j , ; 

5.3.3 Well- Fornicdncss Conditions 

The functional structure is completely constrained bv the constraint equations m the ps rules and die lexical 
entries. 144 Ifkr functional structure must friecJ4rtfefe v^lHHrm(^e^ cmdirwnB: C(>m plctCTic^ . cohcrtnee 
atid consistency . Each lexieafchtrv defines* functional fiJatofrfttShJ?' - ! A '" ! > ( i 



142. We wjll nol discuss the inlcrnal structure of noun phr.tses ut this lime. For now, wc will use the ad hot" predicate 
a-/«oWt'HMurcprvscni the stmclure of \^^ a problem]. 

141 Ttthnipill). U^icaj prc^^cs are np^ aMow^d to icferejicc 1 ,tbc cst^U|f€^Jjl^);,ai;d, surface daughters). The 
BresHan-jCi^Jiin . fbnnii|ajLijun replaces uuf ycojtip with. a ivtw^Ca^jCO^Ic^n^ ^P^f! 

(M/^-a)MiplcniciU);ind;i wc^Ca^coniplcnicnl), ,!,,,, 

144. Sjubieci^vcrh u$feew«n| *■& noj^,d«scribc J d. ; fhere.i%«Jexip| f entry/or each fiirm ; of ..the verb: each asserting a 
different constraint equation on the subject. For example. »p/M5 would havp| rule lilw;: nunrfqib/up)) = {singular}. 
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(262) each slot must be filled (completeness) 

(263) and only those slots may be filled (coherence) 

(264) and multiple assignments tea particular slot must beeoKsistcnt 

Sentences failing to meet these conditions at* ungramtnatical as{265M2£7) IHutfrate. 

(265) *-|hcre is. incomplete 

(266) *lt seems John to be a nice guy. incoherent 

(267) *rhcrc arc a problem. inconsistent 

[Kaplan and Brcsnan80] give an algorithm for instantiating lexical entries; we will not review it here since 
they were not concerned with the same resource limitations. 

5.4 Implementation of Functional Structure 

Examples (268) and (269) illustrate a typical phrase structure rule and a typical texie»lf»rcdicate. 

(268) yap's Notation nrcsnairKapton-liKc Notation 

(dcf-ps-rulc finitc-s s s -> dl : {s-, np-} d2: vp ps rule 

(csubj obi (s- np-) dl = subRup) 

(action (merge down (gct-fsubj up)))) d2 = up 
(chead obi (vp) 

(action (merge down up))) 

(def-pred scem-1 seem 3£cjn -> al : (vp-, ap-} lexical predicate 

(fxcomp obi (vp- ap-) al = xcomp(up) 

(action (subj-control up down)))) subj(up) = subj(al) 

YAP's ps-rulcs and lexical predicates share similar syntax, (269) and (270). Both of them arc CF rules with 
Brcsnan- Kaplan constraint equations encoded into the nonterminals (i.e. <tcrm>). A <tcrm> is defined as 
(271) below. 



145. By convention, all functional slot names will begin with an/ whereas all constituent slot names will being with a c. 
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(269) (dcf-prcd <prcdicatc nainc> <stcm> <tcrm>*) predicate ntle 

(270)(dcf-ps-rulc<ps-rulcnamc><catcgory><tcrm>*) - psrute 

(271)(<role><Omjgatory,OPrional,orSTAK><|^^ term 

Recall tliat YAP's fl//^ operation aut.fmaticalh' advances 4hc • d*f' hi aps-ralc pointer past a n^itcmrinal. 
In addition to updating the ps-rulc, advancing the "dot" also invokes the constraints associated with the 
nonterminal: lliat is, when YAP attaches a daughter to a mother, the daughter is given the <rolci> 1 in the 
mother's frame, and secondly, the <action> field is evaluated with up and down bound to the moUiqr and 
daughter, respectively. 146 For example, when YAP attaches down I to up I in (272), downl becomes the csubj 
ofupl because the "dot" passe* the wufytcravi J- urthennorc, downl becomes Hic^w/y of up 1 because the 
action field specify that down (bound to downl) be mcrfcd wi«h Mp^bOUUdto «p/). ., v , 

(272) [ S J finitc-s->. csubj chcad bejht* attaching 

rsubj: empty 

csubj; iwpiy ) 

= =WALL= = 

Inp-'l ' ■„■-..-. •.,.■■■ 

[ v am] 

ldet dct l 

(273) lg I] finitc-s -> csubj . chead ■■>■■' after attaching 



= =WALL= = 
Idct det l 



«W* [ np . I] 
c^Inp-U 



146. The action field could contain ;m arbitrary I. ISP expression to be evaluated during an attachment, although by 
convention, the action fields merely update functional roles and syntactic features immediately cunueclcd to nodes iu the 
buffers. It is not allowed to violate the FS hypothesis. (It would be an improvement to eliminate the action slot by 
classifying lo possible actions.} ■ n .,...,, ii : ,i ;.■:..;■ 
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'IT>c fstructurc parallels the cstructurc in many ways. Just as we associated a ps pointer With every node, we 
will associate a predicate pointer with every predicate. When a daughter is attached to a ; prcdfcatc; ; the 
predicate pointer is advanced very much like the ps pointer is advanced. Advancing the pointer over a term 
invokes the relevant constraint equations. For example, attaching \$jxcanp tt> swwff, a$ tn figure 11, Invokes 
subject-control. ITiat is, the daughter's understood subject is its mother's subject. 



Fig. II. I *S Attach (revisited) 

sentence: John seems to have left, 
input pointer; 



.Jg. John seems] 



[ vp seems] 



= =WALL= = 
[ to have left] 



Ipunct •' 



finitc-s -> csubj chcad . 

sccm-1 ->. fxcomp 

fsubj:f n p.#BliftJ 

normal-vp -> chcad . (cobj) (excomp) 

sccm-1 -> . fxcomp 

ftubj: l np . John] 

normal-vp- -> ccomp chcad . 
havc-l-> fxcomp. 
fsubj: empty 
normal-x -> cword . 



After attaching, upl's ps and prcd pointers will advance invoking the constraint equations: downl becomes 
upl's excomp and fxcomp, and downl's fsubj is controlled by upl. 



[j John seems] 



[ vp seems] 



( vp . to have left] 



= =WALL= = 
'punct ' 



finitc-s -> csubj chcad . 

sccm-1 -> fxcomp . 

fcubj: [ np _ John] 

normal-vp -> chcad (cobj) (excomp) . 

secm-1 ->ftcomp. r , 

feubj:f np ,JohB] 

fxcomp: L-. to have left] 

excomp: r_ ^o havblclftf 

iH>rmal-vpr-> ccomp chcad, 

havc-l->fxee*npr 

fsubj: [ n __ John] 

nonnal-x -> cword . 



from subject control 
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For another example, thcrc-inscrtion constraints arc enforced when the fob] is attached, using the following 
laical cntcy. for the verb la be. When YAP attaches the/**/, it checks the faubj: if it is the form there, YAP 
enforces number agreement, by merging the num feature of the subject and object. 147 Ilita rule can have 
unbonded consequence* since U^c fsubj can* passed down though an arbitrary number of ra.s.ng verbals 
(like seem and likely). 

(274)(dcf-prcdbc-lbe 

(fobjobl(np-) ;;,.♦,,., 

(action (if 148 ( = *thcrc (gct-fsubj up)) (mergef (gct-fsubj up) down num))))) 

Bmnan-Kaplan's completeness, coherence, and consistency conditions are implemented using the .predicate 
pointers. Completeness is a condition on closing; a node cannot do* until all of its obligatory roles have 
been attached. Coherence is a condition on attaching; a daughter ca»no| attach unless it is an Argument of its 
mother (or controlled by an argument of its mother)." 9 Conf^y is * ; condition on unification; 
inconsistent slots cannot be unified. 



mi Nn„ .ho difference between the mergef and merge functions. The mm merges a particular feature (say num) 
Zl « *Z -ST An Ration ^ u P - <lo»n n^^^^Whtre. only the num feature « 

W 2,Z s uistic notion which distinguishes p^itiot^iingle^ mitMm. "** '* ^ " ^ 

f orms in / Itomlnk, ...). The subject of seem is ^B^^ " «i U*e «-» »»£■ 
Tx£ U 1 wi^pear in that position are not arguments of seem, *fti»***4k. *«**• R>r example. ,n (b) John 



is an argument of nic e. not seem. 

(a) There seems to be a problem. 

(b) John seems to be a nice guy. 
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5.5 An Kx ample 

The cstructurc <»nd Structure for (275) ar? listed bc]aw. This example * very similar to Appendix 2 which 
traces the derivation more carefully. 

(275) The boy was likely to sit? 

(276)CSUBJ:[(NP-)theboy] csiructure 

CrrKAD:[(NP)thcboyJ 
CSPt:C:[(DET)thc] 
CHI-AD: [(N)boyf 
CI1HAD: [(VP) was rtkcly to sit] 
CHHAD:t(V)wasl 
CXCOMP: ](AP-) likely to sH] 
CHKAD:RAP)likclytositl 
CHHA1): [(A) likely] 
CXCOMP: ((VP-) to sit] 
CCOMP:l(COMPJto] 
CHEAD:[(VP)sit] ' 
CHHAD:[(V)sit] 

(277)FSUBJ:[(NP-)theboy] filructure 

FSPKC: [(DKT) the] 
FXCOMP: [(AP-) likcli?tosit] 
FSUBJ:[(NP-)ihcboy] 
FSPKC: [(DKT) the] 
FXCOMlVRVPOtosit] 
FSUBJ:[(NP-)thcboy] 
FSPKC: [(DET) the] 
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6. lexical Transformations 

The traditional argumcrrts ' for complex models (t.g. Inland' AtNi) suggest that simpler mechanisms (Hkc 
YAP) cannot capture the full range oflinguistic generalizations. lTiischapt^rWHIaddVcssflii^brttinsni. 1 * 

(278) "It is well known (cf. [Chomsky64]) Uiat the strict context-free grammar mbdcf fc'rtfrt an adequate 
mechanism for characterizing the subtleties of natural languages. Many of the wpdjtions wl«ch. 
must be satisfied by well-formed Knglish sentences require some degree of agreement between 
dilTcrcnl parts of the sentence which may or may not be adjacent (indeed whjflfymajibc separated 
by a theoretically unbounded number of intervening words). Contcxt-scnskjyc, grammars could 
take care of the weak generation of many of these constructions, but only qtjflic, cos* yf Jogjog U)c 
linguistic significance of the 'phrase structure' assigned by the grammar Jcf/ (PostakHl). 
Moreover, die unaided context-free grammar model is unable to shi>w, ; thc systematic relationship 
that exists between a declarative sentence and its corresponding quft&ioa form, between an active 
sentence and its passive, etc." ; ;j ,.j: ; 

There has always been some controversy over dicsc arguments; currently, GtudajT ^a/da^a^c) leads the 
opposition. Hie confusion stems from two very different interpretations of complexity., 

(279) linguistic complexity : the size of the grammar itself 

(280) compatatbnal complexity : the time and space bounds for an ideal processor 



In general, there is a trade-off between the two types of complexity; u^esiw; of a program (fmguistic 
complexity) is typically inversely related to the power of the interpreter (computational complexity). Woods 
has adopted Chomsky's view that (279) should be optimized at the expense of (2w). 51 Ga?da^s position is 



v:tl 



150. The following quotation is taken from [Woods70J. He is trying lo justify augmenting his ATN model. An 
un-augmcnled ATN (;i Recursive Transition Network RTN) has CF complexity. 

151. Chomsky (personal communications) lias said on many occasions that weak generative capacity (coniputalional 
complexity) is completely irrelevant to the study of grammar. However, weak constraints can be used lo limit the space of 
possible grammars. For example, if language (weak) is actually FS, then no strictly CF grammar (strong) can correctly 
describe Ihc facts. 
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just the reverse. 152 Brcsnan and Kaplan claim that it is possible to ojrtmtffcb^fjth (to have your cake and cat it, 
so to speak). YAP was designed along these lines. It has very minimal computational complexity without 
sacrificing "' linguistic generalizations. this chapter will show how YAP captures many linguistic 
generalizations, greatly simplifying the grammar.' 53 Chapters 6-9 discuss the following topics which arc often 
used to "refute" a position like Gazdar's. 

(281 ) I cxical Transformations (passive, raising, thcrc-inscrion, ...) 

(282) I .ucal Structural Transformations (aux-inversion, deletions, ...) 
(281) Wh-movcmcm (wh-qucstions. relative clauses, ...) 

(284) Conjunction (vpdcte^m,g«ipping,d»pses, ...) 

This chapter will consider ihc following four constructions: other lexical rules arc very similar. 

(285) raising 

(286) it'cxtfaposttion 
(287)passive 

(288) rcanalysis 

There is considerable controversy over these rules; wc have adopted the lexical is t position which "compiles" 
the effect of these rules into the lexicon. That is. there arc different lexical entries for see and seen; see is a 
transitive verb whereas seen is intransitive. Choniskv a<|v,o£atc$ a,ttanslb^a$ion£l position where passive and 
raising arc subcases of move-np. Marcus has encoded Chomsky's analysis in a deterministic framework. This 
chapter will discuss a formulation of Brcsnan-Kaplan lexical rules in YAP's framework. 



1 52. It is widely believed that CF rules arc inherently inadequate (in principle) to describe the fuels. Gaidar (and others) 
give very good evidence lo the contrary. I» is theoretically possible to describe bo* active and passive sentences with two 
different CF rules. Similarly, it is possible to describe yes-no questions with yet another set of CF rules. Since there are 
only a finite number of transformations and only a finite number of base CF rules, one could apply all the transformations 
lo the base, forming a large inelegant (but finite) set of CF rules whkh describe the facts. Gazdar's derivation could be 
viewed as a constructive "proof" that grammar has only CF (computational) complexity. (There are some apparently CS 
constructions to be considered: "respectively" in Knglish. wh-movemenl in Swedish, subject-verb agreement in Dutch 
verbs, and Postal's Mohawk puzzle.) 

153. Gazdar's system has meta-rules to achieve the same goals, though his solution lends to multiply the number of 
grammar rules by a rather substantial constant. Unfortunately, all known general CF parsing algorithms consume time 
pruporlkMial to the size of the grammar, and hence Gazdars solution wpsiow down parsing lime by a rather substantial 
constant. 
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6.1 The I Awical/ Transformational Debate 

The last chapter demonstrated a lexical formulation of there-insertion (coupled with raising). The understood 
subjects were related to each other in the fstructurc by lexical constraint equations. Chomsky would achieve a 
similar result by representing the understood subjects as traces (empty noun phrases) in the cstructure. 
Instead of using lexical constraint equations to bind the traces, he uses a syntactic transformation called 
move-np. 

The differences between these two positions arc very subtle. We will review one argument for each side to 
illustrate the flavor of the debate. Neither of these arguments is definitive; there is a large literature of replies 
and counter-replies. The arguments should demonstrate that competence issues (lexical versus 
transformational) arc orthogonal to performance. The state of performance models is not sufficiently 
sophisticated to distinguish subtle competence issues. It is doubtful whether performance models can ever 
distinguish certain matters of competence. 154 Doth the lexical and transformational positions arc internally 
consistent (for die most part) and equally parsablc (Marcus used a transformational approach). We chose the 
lexical position for its very attractive representation of features (described in the last chapter). Although it 
may be possible to devise a similar scheme in a transformational framework, die lexical representation was 
available when YAP was being designed. The debate has concentrated on two points: 

(289) Do movc-np rules (passive, there- insertion, raising, etc.) leave a trace? 



John was seen. 
Johnj was seen t:. 



lexical 
transformational 



(290) Do infinitives take lexical subjects? 

I believe [ John) [ to be a nice guy] 
1 believe [ s John to be a nice guy] 



lexical 
transformational 



154. An extreme funelion;ilist posiiion might suggest that all competence issues arc ultimately specified by processing 
considerations. This seems most unlikely. 
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'ITic following two arguments debate point (289). 

6.1.1 The Wanna Argument 

The Wanna argument {Brcsnan78] demonstrates that thcrc-inscrtion "must" be a lexical rule since it docs not 
leave a trace (an empty noun phrase in cstructurc). In English, certain verbals <c.g. want, going) dm 
optionally contract with the word to as in (291) and (292). 

(291 ) I want to go home. 
I wanna go home. 

(292) I'm going to go home. 
I'm gonna go home. 

Want + to cannot contract over a trace. Hence contraction is blocked in (293) by the trace of wh-movement, 
but permitted in (294) where the trace docs not intervene. 

(293) WhOj do you want tj to sec BilJ? 
•Who do you wanna sec Rill? 

(294) Whoj do you want to see tj? 
WhOj do you wanna see tj? 

lhc question is: does move-np leave a trace? Is there-insertion a lexical rule as in (295^or a transformation as 
in (296)? If there-insertion leaves a trace, then contraction should be blocked as in wh-movement. But 
contraction is permitted, so there-insertion "cannot" leave a trace. 

(295) There is going to be a movie about us. lexical 
(2%) ITKrCj is going q to be a movie about us. itansfonmlional 

(297) There's gonna be a movie about us. 
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6.1.2 The Away Argument 

[Williams80] argues that the durativc particle away occurs only with intransitive verbs as demonstrated by 
(298)(30l). 

(298) The dial is spinning. away. ; , ; 

(299) *John is spinning the dial away, (wrong meaning) 

(300) John is hitting away at Bill. 

(301 ) Mohn is hitting Bill away. 

He then observes that away can occur with lexically derived intransitives (where there is no trace), but not 
with syntactically derived intransitives (where there is a trace). 

(302) John is eating away. lexically derived 

(303) *Whof is Bill hitting tjaway. syiHaclkalfy derived 

If passive is a lexical rule, then it should allow away by analogy with (302); if it is syntactic (leaving a trace), it 
should block away as in (303). In fact, away cannot occur with passives, so movfi-np "must" lcavea trace. 

(304) *Billj was being hit tj away by Fred. 

Neither position is conclusive. Having adopted the lcxicalist position, wc should show how linguistic 
generalizations can be encoded within the lcxicalist framework. Furthermore, the encoding is subject to the 
processing limitations fftnitc staiie and dctdrminisrri). ' > 

6.2 Raising . ; 

The last chapter illustrated a lexical analysis of raising; wc will summarise the analysis here. There arc two 
types of raising rtrifeS: raising-to-subicct (305) and raising-to-oliflctt i ^8B6».v In both taSCS, there is a raising 
verbal in the higher matrix (e.g. seem, promise, likely, persuade) which determines the type of raising. In the 
seem case (laising-to-subjcct), the embedded subject is bound to the higher sjibjfici; in the persuade case 
(raising-to-object), the embedded subject is bound to the higher object . Brcsnan-Kaplan constraint equations 
elegantly capture both cases. 155 



155. The term luising comes from the old analysis where transformations literally raised the embedded subject up to the 
higher matrix. See [PosUil74] for a defense of the traditional analysis. 



Raising - 95 • Section 6. 2 



(305) subj(up) = subj(xcomp(up)) raising- to-subject 

John seems to be a nice guy. 

John promised Mike to be a nice guy. 

John is likely to be a nice guy. 

John struck Mike as likely to be a nice guy. 

(306) obj(up) = subj{xcomp(up)) raising-to-object 






John persuaded Mike to be a nice guy. 
John forced Mike to bca njecguy. 
John convinced Mike to be a nice guy. 



6.3 Auxiliaries 

YAP analyzes auxiliaries as raising-to-subject verbs; they all select a verbal rxcomp and subject control. 
Unlike raising verbs, auxiliaries select participial Ms 156 features whereas raising verbals generally select 
infinitival tns features. 

(307)f1aas[ xcom pgoingJ. auxiliaries 

'^comp*** 
ijiayiftt^psonej. 

(308) I sgcjrj [ xcornp to go], raising 

Modals (can, may, will, ...) and do select tnsless complements, have takes + en, and be assigns either + ing or 
+ en. 157 For example, the predicate for be would look something like: 



156. The ins feature takes either tense or participle values (since the two have complementary distributions.) The 
possible values are: pres. past, tnsless. + en and +■ ing. 

157. Many analyses separate the two forms of be into an active and a passive entry. Our formulation is more consistent 
with the wait and see philosophy. We claim there is only one copula be which selects an xcomp marked with cither active 
or passive inflection {t+fag. +ev\i fbe mims and passive tutcrprcfcik»w ase cfctetintm*! by Hk partkintes predicate, 
not by the copula. 
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(309)ata-be->al:vp- ! < 

al = xcomp(up) 
tns(al) = { + ing, +en} 
subj(al) = subKup) 

Auxiliaries can nest freely to form sentences like the following: 

(310) T would have been taken. 

(311) I would have been taking the ball. 

Ilierc are a few constraints which limit the possibilities. Medals and W^kPrid participial RnrmsOn their 
auxiliary senses) 158 so they must appear in positions requin.^i present or past inflection. In other words, they 
must be directly dominated by a tensed clause because that i- the only tensed position. For example, (312) is 
ungrammatical because will docs not have a msless form which would normally be required after wMd. (iM) 
is out for similar reasons. 

(312) HwouWwittlwve^ V 

(313) *1 would djj have... ; i : : 

Even with these constraints, the raising analysis seriously over-gencrates. One could fi* this problem jusing a 
small set of motivated features as in [Akamajian79], Curr^tly, YAP will accept serj*e#cps lik£ |U4). It is 
possible that these should be excluded on semantic or pr;i : ;r,atic grounds like (3lS) Which are syntactically 



:';*.-'* \> 



158. Certain niodals are easily mistaken with main verb forms, wh; . !-. have wry different morpho l o gy and dist rib ution*. 

I showtd caw you for that - „■-■•■■■■.■ *« " 

I had the boys take the exam. .; .*:; 

Y4UI&. ■ ■ '• -" 

It isn't dear howa parser can distinguteMhc tvwfenmi YAP hterx-.nc Waited rules to disambiguate a few crtcs. Lexical 
ambiguity is a very hard problem. ■ ' " 
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well-formed, though scmantically questionable. 1 

(314) *l have been having been having... 
*l have had had ... 

0\ 5) ?lt seemed to seem to seem . .. 
?lt is likely to be likely ... 

Hxccpt for this problem, the raising analysis is extremely simple and efficient Sec [Akmajian79J for a critical 
review of these proposals and some alternatives. 

6.4 It-extraposition 

The raising analysis has a number of manifestations; it has played a crucial role in there-insertion and 
auxiliaries. It also turns out to be important in iMsxtraposftion, illustrated by (316H318) below. 

(316V It was believed that! would go. 

(317) It was promised that I would go. 

(3 18) U seemed likely Ujai I would go.. 

It-extraposition is similar to there-insertion; both cases illustrate a dependency between a subject and a deeply 
embedded constituent. In there-insertion, the "dummy" form there depends upon a deeply embedded flflyji 
phrase such as a problem; in it-extraposition, the "dummy" it depends upon a deeply embedded clause . 



159. We could suggest some more fillers to exclude some of the additional cases. For example. Have doesn't take +ing 
in its auxiliary form. 

I have taken it 

*l was having taken it 

A second condition blocks two adjacent verbs with + wig inflection. 

*[... +ing+ing...] 
* I am being being „. 

These filters are merely descriptive; a true theory would explain these facts. 
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(3 19) There seemed likely to seem likely ... to be a. problem . 

(320) U seemed likely to seem likely ... that I would po. 

YAP uses a similar mechanism in both cases: just as there arc lexical entries which check their fcubj slot for 
the form there, there arc lexical entries which check for //. Since subjects can ^bc raised arbitrarily far, 
it-extraposition can have unbounded consequences. 160 

(321)(dcf-pfredbc-tbe 
(fobj obi (np-) 
(action (if( = *thcrc(gct-fsubj up)) (mergef (gct-fsubj up) down num))))) 

(322) (def-pred likely- 1 likely 

(fceomi»oW(s-) , .• ■-->'!■•. .- . 

(actionXif < *» *it (get-feubj ufcW (mw iypf^Wld^Mbi jr 

The form // in (323) is co-indexed with the scomp (sentential complement fa distinguish it from the 
pronominal it in (324). The two interpretations have different semantics.^ 

(323) It seemed that we were nice. (meaningless //) 

(324) It scorned to be nice. - >,: (pronomtnatia) 

Similar comments apply to there; (325H326) demonstrate the different semantics of there. 

(325) There was a problem. (meaningless there) 

(326) I went lhej£. (pronomial there) 

6.5 Passive 

Our passive analysis depends on the formulation of auxiliaries as raising verbs. Passive participles do not 
stipulate the auxiliary. It happens that to be is the only auxiliary that can take a passive participle. 1 ** This is 



160. Note lhat it-extraposilion merges ever)' feature associated with the subject whereas there-insertion only merges the 
num feature. Hence, il-cxlrap«>sitioii uses the merge function whereas there-insertion uses the /?KT£</functk>n. 
16J. Except for haw. all other auxiliaries block ^enpktii^pkei r^y rmn^ i sO^Uheri]rjijfCaturc with their fxcomp .) 
For some unexplained reason, hive blocks passive interpretation of its fxcomp. 
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purely accidental; passive participles arc found in many other constructions without the verb to be. The 
verb to be is identical in both (327) and (328); the difference is restricted to the participial phrases seeing me 
and seen. 

(327) John was seeing me. 

(328) John was seen. 

There arc two lexical entries, one for seeing (329) and one for seen (330), which are related by a lexical 
redundancy rule to capture the passive generalization. 

(329) active-sec -> al:np- a2:np- 
al = subjdip) 

a2 = obj(up) 

(330) passive-see -> al:np- 
al = subj(up) 
tns(up) = -fen 

In the Brcsnan-Kaplan framework, all lexical entries are "tried" noh-dcterministically; structures meeting the 
functional well-formedness conditions (coherence, completeness, and consistency) arc considered valid 
interpretations. This is a perfectly reasonable competence model; however, it may have two problems as a 
model of performance: 

(331) very large lexicon 

(332) non-determinism 



162. Here arc three constructions involving passive participles: 



a fallen leaf 

He seemed persuaded to leave. 

I saw a horse taken past the barn. 

There is a considerable literature discussing passive generalizations: our fomuilation is consistent with the lexical analyses, 
although many of the details have not been implemented. 
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YAP uses a virtual lexicon to alleviate problem (331). Instead of storing all the lexical entries literally in a 
huge array, YAP stores only the core entries; other entries arc generated upon demand. Viewing the lexicon 
as a black box, it shouldn't be possible to distinguish the real entries from the virtual ones. Ihc virtual lexicon 
is very analogous to virtual memory systems which page address locations into real memory upon demand. 
'ITicse schemes take advantage of a space/time trade-off. 163 

Determinism is more difficult to arrange. Mow can YAP decide which lexical entry to use? The lexical 
ambiguity problem is extremely difficult. In this case* there arc some fairly good heuristics. The unmarked 
case is triggered by a + en morphological feature, though there arc several marked rules to disambiguate some 
of the more difficult cases. ITicse rules may seem ad hoc, but they do have to be -stated' in one way or another. 
Perhaps we will find an explanation someday; for now, we will make do with a descriptive tfKtory. 

(333) John was seen. (the unmarked case) 

(334) John has seen Dill. (perfect construction) 

(335) The horse raced past the barn. (+en/+ed ambiguity) 
The horse raced past the barn fell. 

There arc two exceptional cases: the perfect construction (333) and the + en/ +ed morphological ambiguity. 
The perfect construction blocks the passive rule from applying to its complement This fact is stated in the 
lexical entry for have. The morphological problem in (335) is disambiguated by the unification procedure. 
The two senses of raced ({ +en, past}) arc merged (intersected) with the two senses of a tensed clause ({pres, 
past}) producing a unique result (sec figure 12). 

YAP has a production rule to generate a passive predicate pointer when it is needed. It looks something like 
the following, although a number of details have been omitted for clarity. 



163. Page faults (generating lexical entries on the fly) become less and less probable as. more and more lexical entries arc 
added to the core lexicon. It may be more efficient to include redundant information in the lexicon which is frequently 
accessed, thus reducing the chance of a page fault. In other words, it may be worthwhile to sacrifice some linguistic 
complexity to achieve improved computational complexity. 

164. For example, there has to be a mechanism to prevent the ni|e from re-applying arbitrarily often to the same 
predicate. There is an uninteresting lisp expression in the pattern to accomplish this. 
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Fig. 12. Disambiguating +cn/+cd 
sentence: The horse raced past the ... 

[ s the horse] tns: {prcs, past} 

= =WAI.L= = 

t raced) tns: {past, + en} 

fpPast] 

idct^l 

mere is a constraint equation which unifies a clause with its head (the vp). Whence head is attached the 

constraint equation is evaluated, disambiguating the iQS features. Ilic two senses of raced {{ +cn, past}) arc 

merged (intersected) with the two senses of upl ({prcs, past}) producing a unique result 

[ s the horse raced] tns: {past} 
[ raced] tns: {past} 

= =WALL= = 
[p Past] 



(336) (dcfrulc passive trans 
(pattern ()(=+cn)) 
(action (passivizc-prcd downl))) 



The function p^sivi/c-pred transforms downl's active predicate pointer into a passive one. (It simply 
replaces the fsubj slot with the fobj slot.) 165 This should have the same external Appearance as though there 
were passive predicates stored in the lexicon. It is merely a space/time trade-off. 



6.6 lleanalysis 

In general, prepositional objects do not passivize. For example: 



165. Unfortunately, this docs require copying the predicate pointer. 
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(337) The ball was gone to. 
"^ITic river was seen at 

The boy was taken the ball from. 

However, there arc some marked cases where passive is possible. To account foe these facts, it lias been 
proposed that certain verb-particle combinations (c.g. arrive at and look at) can reanalyze into a single verb 
complex. The reanalyzed form (338) can passivize, unlike (339), because the solution is a verbal object 
whereas the station is prepositional object 

(338) ITicy { v arrived at] f np . the solution]. 
The solution was arrived at 

(339) They arrived L_. at the station]. 
The station was arrived at 

Since YAP is not capable of distinguishing the semantic difference between the solution and the Mfltfieftiit 
cannot distinguish <338) from (339), When syntactic clues arc sufficient as in (340>(34l), YAP correctly 
performs the reanalysis. 

(340) I looked at the picture. 
The picture was looked at 

(341) I went to the batt. 
The ball was gone to. 

The difference between look and go is stated in the lexicon; look reanalyzes with at, but go docs not reanalyze 
with to. Hie lexical entry for look at is listed below. Notice that it takes a direct object, not a prepositional 
object 

(342) (def-pred look-at-1 look 

(fsubj obi (np-)) 
(fcasc obi (p) 
(fobjobl(np-))) 
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We have seen how a number of lexical rules (raising, it-cxtraposition, there-insertion, auxiliary formation, 
passive, and reanalysis) arc formulated in YAP. This shows that many of the generalizations can be captured 
by a relatively simple device. 
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7. Local Structural Transformations 

The last chapter demonstrated several rules which operate on predicate pointers (("structure). 'This chapter 
will discuss structural transformations which operate on constituent structure (cstructure). There arc some 
important differences between lexical and structural rules. 

(343) Lexical rules arc local in /structure; structural rules arc local in cstructure. 

(344) Structural rules have no lexically marked exceptions. 

(345) lexical rules are structure preserving. 166 

My these criteria (which arc admittedly very pro-lcxicalist), it is very hard to find suitable candidates for a 
structural rule. (343) is not very discriminating; as we have seen, it is generally possible to state many rules in 
either the fstructurc or the cstructure. (344) is very pro-lcxicalist, since almost every linguistic generalization 
has an exception. Only (345) establishes a class of structural rules; some rules (e.g. root transformations) are 
not structure preserving. This section will analyze two root transformations: aux-invcrsion and imperative . 

The structure preserving property [Hmonds76] is analogous to side-effect 168 free (applicative) programming; 
both moves attempt to establish an invariant representation which remains intact after an arbitrary number of 
transformations (function calls). Linguists have found the invariancc notion to be useful for describing 
grammar; computer scientists have discovered invariance important in program verification. It is generally 
agreed in both fields that structure preserving (applicative) formulations arc desirable. 



](>(). [F.monds76] postulates that transformations divide into two categories: Structure-Preserving Transformations and 
Root Transformations. The former introduce or substitute a constituent (' into a position in a phrase marker held by a 
node (': root transformations move, copy and insert a constituent in root clauses. 

167. Actually the case is not so clear: there may be ways to reformulate these transformations to be structure preserving. 
For example. [Kaplan and Urcsnan SO] present a structure preserving analysis of imperative. 

168. A program is said to cause side-effects if it modifies data structures in a non-inverlible fashion. In general, it is 
possible to avoid side-effects; there is a school of computer scientists who advocate completely side-effect free 
programming. This position is somewhat analogous to the lexicalist schcxil of linguists who advocate side-effect free 
analyses. 
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7.1 Aux-invcrsion 

Perhaps the best example of a structural transformation is the so-called aux-invcrsipn rule which has applied 
to(346)-(350). 169 

(346) Mave I taken the ball? 

(347) Which balls have I taken? 

(348) Never have \ taken so many balls! 

(349) Under no circumstances am 1 permitted to release these documents. 

(350) Nowhere could he find an alpaca carpet 

YAP's aux-invcrsion rule undoes the inversion by switching the buffer cells containing the auxiliary and die 
subject noun phrase, thus capturing the linguistic generalization without increasing the computational 
complexity (memory is still severely bounded). The aux-invcrsion rule inverts down I and down2 as 
illustrated in (351). It also labels upl with the mood feature {wh-q, yes-no-q} to distinguish the sentence 
from its declarative form. 170 

(351) sentence: Have I taken the ball? 
input pointer: the ball? 

before after 

= =WALL== ==WALL= = 

t v havc] [ n p-H 

[ np . fl I v navel 

l v taken] [ v taken] 

A simple form of the aux-invcrsion rule is shown below. 171 



169. Only yes-no and wh-queslions have been implemented: the other cases shouldn't be too much more difficult 

170. This doesn't work in the proposed adverbial case. Never haw I seen so many bulls! Bob Berwick (personal 
communiailions) has suggested that the inverted forms share a common l.F (logical form) interpretation whkh 
distinguishes (hem from declarative sentences. 

171. The last term of the pattern could be an arbitrary lisp predicate which must be truejn order for the rule to match. 
In practice, the predicates tend to test features of nodes in the buffer. In this case, the predicate crole-can-advance? is 
testing if upl is looking for a subject. Some details have been suppressed for clarity. For example, there are some 
agreement constraints which will be discussed later in this chapter. 



Aux- inversion - m - Section 7.1 



(352) (dcfrulc aux-invcrsion trans 

(pattern ( = root) ( = aux verb = np-) (crolc-can-advance? upl csubj)) 
(action (invert) (setfeat upl (yes-no-q wh-q) mood))) 

Aux-invcrsion is possible when upl contains a root clause 172 looking for a subject, and the lower buffer holds 
the inverted auxiliary/np- pattern. 173 ITiis rule was taken almost directly from Marcus' Parsifal. 

7.2 Imperative 

Imperative is a deletion rule which applies to root clauses. 174 The parser Simply restores the deleted elements 
and finishes the sentence as if nothing had, been missing, Given,? scnjejicc lie (353), YAP will insert the 
words you will into the lower buffer, undoing the imperative transformation. YAP will finish the sentence as 
if it had been parsing (354). As in aux-invcrsion, the tr^sfo^aUonjdds a^ood, feature to distinguishes, the 
transformed sentence (353) from the uiitransformcd sentence (354)^ The rulcisgiven as(356)^elow; 175 



172. The highest clause is a root clause. There are some other instances of root phenomena which YAP does not 
currently handle. For example. I said "what are we eoinf to do!T ! 

173. The following verbs act as auxiliaries in English: be. hare. do. can. will. may. shall, muM, and perhaps a few others. 
There is another marked rule (described in the next section) which blocks aux-inversion when do and have (in American 
English) are being used in their main verb senses as below: 



Have the boys take the exams! (main verb) 
Who had the boys take the exams? 
Doit! 
Who did it? 

Have the boys taken the exams? (aux verb) 

What have the boys taken? * . 

Did il bother you? 

Who does it bother? 

It is an unexplained fact that be and the British use of have invert (even in the mainverb sense). 

174. [Kaplan and BrcsnanHO] give a lexical analysis of imperative. 

175. This rule was also taken from Marcijs" Parsifal. There is one d|ffcrcnc£: his rule drops the word^ww into the buffer, 
not the words you will. YAP will parse (a]f like (b)- Rtrsifiil wffl p^ 

(a) Re good! 

(b) You will be good! 

(c) *Yoti be good! (wrong meaning) 

YAP drops the will to absorb the tense constraint on root clauses; root clauses are tensed, except for imperatives which 
have no overt tense marker. 
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(353) Take the ball! 

(354) You will take the ball! 



(355) before 






after 


U 






u 


= =WALL= 


= 




= =WALL= = 


l v takej 






t np .youl 


[ dct thc] 






l v will] 


f n ball] 






{ v take] 

Idct^l 
[ n baH] 


(J56) (defrulc imperative 


trans 




(pattern ( = 


= s)( = 


= v) (and ( = 


= tnslcss 176 down 1 ) (crolc-can-advancc? uol csubj))) 


(action (setfeat i 


jpl imperative mood) (drop-words you will))) 



7.3 Differential Diagnosis 

It happens that both aux-inversion and imperative have very similar patterns. In examples like (357)-(359), 
there is some difficulty deciding which u-ansformation should apply. Some cases, such as (359), are 
grammatically ambiguous, and hence, it is not possible to disambiguate using just the rules of grammar 
(competence). 177 

(357) Have the boys take the ball! imperative 

(358) Have the boys taken the ball? inversion 

(359) Have the eggs fried ... ambiguous? 

A non-dctcrministic system could "try" both rules, accepting aH analyses that happen to work out A 
deterministic system is posed with a difficult problem; both transformations (aux-irtversion and imperative) 
cause side effects which cannot be undone A deterministic machine has to make the right decision the first 
time; there will be no recovering if it selects die wrong transformation. 'ITiis section will discuss procedures 



176. The predicate =/»i/m tests for null inflection. 

177. The ambiguity may not be realized in performance. Marcus claims there is a strong preference for inversion in the 
unmarked case, though the marked interpretation can be forced by. semantic and pragmatic biases. 
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for deciding which transformation should apply. 

Marcus believes this problem results from a lexical ambiguity between the two senses of have. m The 

* vy..d:' : 
auxiliary hove undergoes inversion as in (360) unlike the main verb have (in American Knglish). Hence, if we 

could distinguish the two forms of have, we could decide which transformation should apply. Marcus invokes 

a marked rule (360), called llavc-diag . to disambiguate difFcrciMially 179 between the two senses of have. 

(360) pattern: 

downl: [ v have] 
down2: [ <any> ] 
down3:[ <any> <any>l 

If down3 is tnslcss or down2 is plural (first or second person), marked exception 

then run imperative next lM 
Otherwise, run aux-in version next unmarked default 

,...'* -■-"(■ 

■ .-■ i'z'i:\, ' ■ ; . . " '■ * '■ ■" ■ '■ " 

Hie default path (inversion) is taken, unless there is marked evidence to the contrary. Marcus claims that the 
marked information must appear in the next three eonstHuents. Ws has sw empirical evidence indicating 
that many people cannot disambiguate (361 M362) bccajusc^crc is no disambiguating information within the 
specified lookahead. (n (363)-(364). the c^^lMnteippeteUwiinvs^^ 
words, and hence, (363H364) receive the exceptional interpretation (imperative). 



178. The main verb seme of have m verts more frcctym British English! 
American; Dp you have ^ match? 

British: Have you a match? 

179. The term differeiWal diagnosis was derived from medical appticalioiis. It Is bclhrved that doctors have precompiled 
rules to differentiate between medical conditions which liavc similar symptoms hut rehire, ycry diflf erent diagnose 
[Davis77J refers lo these rules as meta-rules because they reason about mles. this is a very powerful technique, though 
potentially expensive. 

18(1. Actually. Ihis rule has a slight flaw; it fails to distinguish Have [eaten? Irom Have ms. eaten!. This suggests that case 
features (in addition to person/number) should be used lo disambiguate. NcUhcr^YAP no/ Parsifal use reflexive features 
lo disambiguate. For example, compare: 

Have yourself completely taken advantage of. for all I cane! 

Have ion completely taken advantage of every chance? 
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(361) [j Have] [j the packages] [^ delivered] tomorrow. unmarked 

(362) [i 1 lave] [2 tlic soldiers] [1 given] their medals by their sweethearts. 

(363) I lave them delivered tomorrow. marked 

(364) Have the soldiers take their sweethearts to the dance. 

This approach works in a large number of cases. Like other marked rules, it suggests three important 
questions: 

(365) How arc diagnostics restricted? 

(366) Is there any empirical support for this approach? 

(367) 1 low many diagnostics will be needed? 

Marcus' lookahcad buffer addresses question (365). The three constituent limit is consistent with the 
empirical evidence mentioned above (361 )-(364) and the garden path phenomena. Although Marcus' 
approach has these desirable characteristics, there is some concern that a complete grammar would require 
too many diagnostics. Diagnostics arc used when there is a lexical ambiguity that would lead to multiple 
cstructurcs. The number of diagnostics becomes troublesome when they compare two or more 
transformations at a time, and hence, there may be a combinatory number of diagnostics. It is quite 
reasonable to place conditions on a transformation one at a time; the problem comes when multiple 
transformations must be compared differentially. It is possible that differential diagnosis may require an 
inordinate number of rules. We will reformulate Marcus' Havc-diag as follows: 

(368) Aux-inversion is blocked when any of the following conditions cannot be met: competence 

down 1 has pres or past inflection 

downl can take down2 as subj (agree in person, number, gender and case) 

downl can take down3 as xcomp (agree in inflection) 

(369) Imperative is blocked when aux-inversion can apply. performance 



181. Like other performance limitations, the buffer length is subject to a certain amount of individual variation. 

182. We accept Marcus' assumption that 110 rule can access beyond down.3. although additionally, we allow rules to 
access npl, up2 and up3. This is a performance limitation on backup/lookahead. It seems to be subject to the same 
idiosyncratic behavior that plague other performance constraints (e.g. individual variation). 
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Our formulation has three advantages over Marcus': 

(370) Clear separation of competence and performance 

(371) Covers a wider range of cases 

(372) Fewer differential rules 

It is important to separate competence and performance; performance filters such as (369) arc generally more 
idiosyncratic than statements of competence (368). Performance phenomena arc often subject to semantic 
and pragmatic biases, garden path behavior and variation from one informant to another. For example, (369) 
is subject to a certain amount of individual variation as Maim has observed; it is unlikely that (368) can be 
overruled in the same way. 

Our statement is more general than Marcus*; His rule only applies to hdve\ our formulation covers all 
auxiliaries, including didait&wasa& illustrated in (373H376). 

(371)Whjidislit? no inversion 

(374) Who djd it bother? inversion 

(375) WJio.wjs.it? no inversion 

(376) Who w|§ it bothering? inversion 

Thirdly, our formulation requires fewer differential diagnostics to disambiguate between several 
transformations. These rules are particularly costly because the number of necessary rules grows very quickly 
with the number of transformations. We have foctorcd the agreement constraints from the differential 
diagnostics. Modularity is a welcome step. 

It would be desirable to completely eliminate differential diagnostics, rules that mention multiple 
transformations. Wc will propose an alternative formulation that achieves many of the same results without 
the undesirable cost associated with mentioning multiple transformations in a single rule. Traditionally, 
transformational grammarians imposed ordering constraints to block one rule when another can apply. 
Marcus' scheme is less restrictive than die traditional ordering constraint: he imposes a partial order instead of 
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the more standard lolal order. J 

Unfortunately, ordering relations are very difficult to formulate, as standard transformational grammarians 
have discovered. There always seems to be an ordering paradox. An alternative formulation expresses the 
ordering relation in terms of features. 184 Suppose that imperative requires more precisely determined 
features than aux-invcrsion: it cannot trigger while the ins features (for example) arc undcrdctcrmincd. 
Aux-invcrsion is less restrictive; it will trigger as long as the ins features arc compatible, whether or not the 
other possibilities have been excluded. This will assure that aux-invcrsion takes precedence, without 
explicitly mentioning both rules in the same diagnostic. 

The ordering mechanism is illustrated in (377)-(378). =?lns tests for a pres or past feature, disregarding the 
other ins features; ---insless tests for an uniquely determined insless feature. A word like have, which is both 
pies and insless ({pres, tnslcss}), passes the aux-invcrsion pattern (377), but fails the imperative pattern, and 
consequently, aux-invcrsion will be given first crack. If it should be explicitly blocked (by an agreement 

185 

constraint), then imperative will be given a chance. 

(377) (dcfrulc aux-inversion trans 

(pattern ( = root) ( = ?tns = np-) ...) 
(action ...)) 

(378) (defrule imperative trans 

(pattern ( = root) ( = tnslcss = np-) ...) 
(action ...)) 

In this way, YAP achieves the effects of differential diagnoses without the associated disadvantages. There is 
a natural separation of performance and competence. The competence idealizations specify agreement 
constraints; the realistic performance model qualifies them with "ordering" relations. We have proposed a 
statement of the "ordering" relations which may be more robust than conventional formulations. 
Nevertheless, the rule ordering problem would completely evaporate if YAP had lexical (side-effect free) 



183. A total order is transitive, reflexive, and antisymmetric; every element is ordered with respect to every other. 
Marcus used a partial ordering scheme (priorities). A partial ordering scheme is not antisymmetric; two elements may 
have the same priority (unordered). 

184. This idea is only partially implemented in the current version, which still contains some differential diagnostics. 

185. The ins feature is disambiguated when inversion is blocked. 
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formulations of these transformations. Sidc-cffccts should be avoided whenever possible, especially in a 
deterministic framework. 

This chapter has outlined ..an approach for capturing total structural transformations, taken from Mitrcus' 
Parsifal. YAP undoes the transformations by man»puJat«\g-the bokahcad buffer. Wc hay<c discussed two 
structural transformations and their interactions. Sinc« k is possible U) implcmeiu all of Marcus' 
transformations in this framework, a simple device is adequate for capturing many linguistic generalizations. 
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8. Wh-movement 

A number of long distance transformations are categorized under wh-movement including: wh-qucstions, 

186 

embedded questions, relative clauses and topicalization. 

(379)WhOjdidyouscexj? wh-question 

(380) 1 wonder whOj you saw Xj? embedded question 

(381 ) I saw a boy whOj you know x y re,alive clause 

(382) The ballj. Bill took Xj. lopkalizalion 

ITicsc constructions arc particularly interesting because the trace (Xj) can be arbitrarily far from the operator 
(whOj). 

(383) WhOj did Bob say that Bill said that ... Mike said I saw Xj? 

(384) I wonder who. Bob said that Bill said that ... Mike said 1 saw Xj? 

(385) I saw a boy whoj Bob said that Bill said that ... Mike said 1 saw x } ? 

(386) l"hc ballj. Bob said that Bill said that ... Mike said I saWj? 

Wh-movemcnt illustrates yet another dependency across seemingly unbounded distances. Like 

there-insertion, the solution is to find a representation (fstructurc) where the dependencies arc local. YAP has 

ffti 
another grammatical role (fwh) to hold the wh-clcment 

(387) Thcrej seems xj likely Xj to seem Xj likely - move-np 

(388) WhOj did Bob say that Xj Bill said that 34 ... move-wh 

There are understood fwh elements in (388) just as there arc understood fsubj elements in (387). The binding 
relation forms equivalence classes in both cases. The equivalence property is very convenient for 
computational reasons discussed in chapter 5. All the co-indexed elements are represented collectively as a 
single node, not once for each individual member. Consequently, wh-movemcnt is bounded in fstructurc, 
even though it appears to have unbounded consequences (see figure 13). 



186. Many people object to the topicali/ation construction. 

187. Our fwh role is like Brcsnan-Kaplan's super-down register, Chomsky's conip node. Marcus - wh-comp feature, 
Woods - hold cell. Although these mechanisms are similar to one another, they do have slightly different properties. For 
example YAP's fwh role is passed from phrase to phrase whereas the other mechanisms pass ttie clement from clause to 
clause. In this respect. YAPs approach is more like [Kpstcr78j and (Gazdar79a.b.c] which treat all nodes equally; there 
are no special bounding properties associated with clause nodes. 
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Fig. 13. Wh-movcmcnt 

Who l did Bob say that Xj Bill said . 

Xj prcd: who 

x-2 prcd: do 

fwh:xj 

tns: {past} 
fsubj: xi 
fxcomp: x^ 

Xj prcd: Bob 

x^ prcd: say 

fwh: Xj 

tns: {tnslcss} 
fsubj: xj 
fscomp: x^ 

xj prcd: say 

fwh: xj 

tns: {past} 
fsubj: Xg 

x* prcd: Bill 



There are some differences between move-np and movc-wh; move-np uses lexical (predicate) rules to bind 
the intermediate subjects whereas movc-wh uses structural (ps) rules toi bind the mtcrrriedtate fwh slots. 
Compare (389) and (390); 188 Move-wh is a structural rule because it is constrained by phray ; s {ruc t ,urc ale 
such as (390), whereas movc-np is lexical because it is constrained by prc ^H ^tf fi rujefl as in (389). 



188. It is possible to represent these rules much more efficiently using a murkedness theory. For example, the head is 
unified with its mother (by x-bar theory) unless explicitly marked otherwise. 



Wh-nwvemenl 
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(389) sccm-1 -> cxcomp:{ap-, vp-} 
cxcomp = fxcomp(uD) 

fsubj(up) = fsubj(fxcomp(up)) yove-np 

(390) vp -> chcad:v (cobj:np-) (cxcomp:xp-) 
chcad = up 

cobj = fobj(up) 

cxcomp = fxcomp(up) 

fwh(up) = fwh(fiicomp(up)) move-wh 



8.1 Island Phenomena 

Wh-clcmcnts cannot be extracted from just any phrase; there arc certain "islands" which are opaque to 
wh-movcmcnL Islands arc be explained in terms of consistency and coherence in the Brcsnan-Kaplan 
framework. Some extractions arc blocked because the fwh slot is already filled Ohconsistent) and some are 
blocked because there isn't a slot to fill (incoherent). 

8.1.1 Wh-islands 

In general, there can only be one extraction from a phrase because the fwh slot only has room for one value; 
multiple values will be inconsistent. Hence the following jwn^esar^uiigrarnrnatical because there are 
inconsistent fwh elements associated wju) the bracketed expressions. 

(391) *WhOj docs John wonder [where Bill saw tj]? 

(392) *Whatj did you ask me [whei-c you could buy tjl? 

(393) *Whati did [who see qj? 

(394) * I wonder what j [who bought tj]? 

(395) *Whatj docs John wonder [where to put tj]? 
(3%) *Whcrej docs John wonder [whe to put tj]? 

(397) *Whatj docs John wonder [to put tj where]? 

(398) *WhcrCj docs John wonder [to put what tj]? 



189. These examples were given in Ken Hale's 1979 fall class at MIT- 
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There arc some wh-islands which allow extraction. We have no explanation for Ais fact; YAP cannot 
currently parse wh-island violations. This is a very marked phenomenon wHteWttiight be'covcVcd by a 
marked rule. 190 u ; , ^)^ 

(399) ?What docs John know how to do? 

(400) ?What did John ask how to cook? 

(401 ) ?Hcrc arc the books that I don't know what to do with? 

(402) ?l just read a book which I can't figure out why anyone would write. ? 

(403) ?l like the girl that you wonder what John sees in. 

(404) ?l found the book that John couldn't remember what the title of was. 

8.1.2 Ross' Complex NP Constraint 

[Ross67] observed that extraction is generally blocked by np- brackets, as in (405H4Q7). (Yhis is an over 
simplification.) 



(405) *Who | 

(406) *WhO| 

(407) *Who: 



do you know [ the man that married tjj? 
did you hear [ a rumor that John betrayed tjj? 
did you find ( np . a copy of a photograph of 1$ 



YAP expresses these facts in thenp- pr-rtrte. Most ps-rutes pass Mjwh clement though constraint equations. 
For example, the vp ps-rule has a constraint equation ! tb 1 pfe i&6 jWi element into its jrcomp: 
fwb(up) = fwh(xcomp(up)). There is no such rule associated with np-. Hence, an attempt to move an fwh 
clement over an up- bracket will be incoherent This account fi# ttKtr)jnpal contrast between (408)-(410) 
and the examples above. <•■, 

(408) Who do you know that John married? ■.;-..-.■ a- ..,,. , 

(409) Who did you hear that John betrayed? 

(410) Who did you find? 



190. These sentences were given in a recent talk by George Hart at MIT. Some informants find these sentences perfectly 
acceptable while others (including the author) find them extremely-marginal. 



Rvss' Complex NP Constraint 
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There arc some more difficult cases. For example, if extraction is blocked by tip-, then why is |411) 
grammatical? YAP has a marked rule to cover this case. These picture noun phrases arc still problematic for 
linguistic analysis. ITic answer appears to involve the specificity of tftc up-. 

(411) Who did you sec [ np . a picture of t]? 

(412) *Who did you sec [ n __ John's picture oft]? 

An account has been provided for both types of islands. We do not claim that these facts follow from YAP's 
design. Our position is much weaker; we merely claim that these facts arc compatible with the design. Many 
linguists arc currently working on a more explanatory theory. 

8.2 Gap Finding 

The really hard problem with wh-movement is finding the "gap* where the wh-clement originated. This is 
not particularly difficult for a non-deterministic compctchcc theory, but it is (probably) impossible for a 
deterministic processing model. YAP has made some simplifying approximations to the competence 
idealization which may be valid in a realistic performance model. In an ideal non-deterministic framework, 
there could be a phrase structtrte rule like: 

(413) up:gap-np- -> dl:t 
fwh(up) = dl 

Unfortunately, it is very difficult to formulate this rule in a deterministic framework. YAP approximates the 
ideal competence by looking for a gap after the other default ps actions have failed. Find-gap is a new 
dcfault-ps action which is applied after the other actions as in (414). 

(414) attach 
predict 
close 
find-gap 

This heuristic favors the latest possible gap. It corresponds to Fodor's \ ^st-Rcsort Modfil fif Gafi Eiadiflg 
fFodor78]. As she correctly observes, there are some problems wkh this model. Like other marked 
exceptions (sec chapter 3), there arc some marked rules to handle the problematic cases. Before suggesting 
some modifications to save the last-resort model, it would be useful to consider some alternatives. Fodor 
proposed three models of gap finding (415)-(417) and ultimately settled on the third alternative. 
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(415) First-Resort ([Marcus79l) 

<416)lxist-Rcsort(YAP) 

(417) Lexical Kxpcctation/Arc-Ordcri^g(| t K|i|jlan72].iF©dpr78J) 

The first-resort and last-resort models can be implemented by thp default ps actions. The first-resort model 
orders find-gap first whereas the last-resort model orders it last 



(4i8)EkstKssQri 


l^st-Rcs 


find-gap 


attach 


attach 


predict 


predict 


close 


close 


find-gap 



The first-resort and last-resort models do not exclude lexically marked cases; they merely suggest an 
unmarked default. In some sense, the arc-ordering strategy denies structural correlations; it explicitly, lists the 
preferences for each verb and hence it would be optimal Just in case the various structural possibilities were 
randomly 191 distributed throughout the lexicon. 192 We believe there is a strong bjas in favor of (41$), 
although it may be overruled by lexical marking in certain cases. Let us consider some evidence: 193 

8.3 Evidence for the Last- Resort Model 

(419) I gave the boy who you wanted to give the books to three books. 

Sentence (419) is unacceptable. 194 Grammatically speaking, it is extremely ambiguous; there are no less than 
four possible gaps as shown in (420). 



191. A set is random when Ihe shortest description explicitly lists each of its members. 

192. Arc-ordering is often formulated within a depth first (DFS) control structure. The DFS is in fact imposing a 
structural constraint; it encourages low attachment. In Marcus' non-deterministic framework, these structural correlations 
have to I* staled elsewhere. The de^^ ; 

1 93. Possjhte gaps are shown in pi#ei»thcscs. Plus. (■+ ) and mimis (- Hndicate relative processing difficulty. The mow 
acceptable of the pair are marked with a plus. 

194. Of the 40 test sentences in fMarcu*79. Appendix D|, this is tbeonly One thtrfVftl* cannot parse. (Some informants 
find the last (jap acceptable as in: . I give the boy (who you ivanled, (a.yxe $f t books lo l} three books. This strategy is not 
incompatible with the I jst-Resort Model, although it would require a slight modification.) 
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(420) # I gave the boy who you wanted (t) to give (t) the books (t) to (t) three books. 

Why is it so difficult to find to find the gaps? The last-resort model prefers to attach lexical material over gap 
finding and hence it misses all the gaps. litis unacceptable sentence is very supportive of the last-resort 
model but rather damaging to the first-resort model which can easily (?!) find the first gap. The examples 
don't need to be so extreme. Wc have already seen a garden path sentence (421) also favoring the last-resort 
model. (422) shows that these GPs arc fairly productive. 

(421) # I told the boy the dog bit Sue would help him. 

(422) ??I called tlic guy who tlic car was smashed up by a rotten driver. 

Corollary (423) 195 immediately follows from the last-resort model: np gaps arc extremely marked 196 in 
positions immediately before lexical noun phrases. Tlic reason should be obvious; the last-resort model 
prefers attaching the lexical noun phrase over creating the gap, unless there is positive evidence (i.e. semantic 
clues) to overrule the defauk. Thiscorollary accounts for the badness OfiW21.) and (422), Two of the possible 
gaps in (420) are also excluded under this corollary to the lasfriraort strategy. 

(423) TJi£ Tracc-NP Corollary : In the unmarked case, #[... tj NP ...], where q is bound to a noun 
phrase. 

This corollary correctly predicts preferences n double object constructions. The lexical noon phrase is 
generally interpreted as the first object unless there is positive evidence to the contrary. Even then, the 
marked interpretation is generally less acceptable. 

(424) + What did I give the boy t? 

+ Who did I give the book to t? 

- Who did I give t the book? 

(425) + What did you call a drunken sailor t? 

- Who did I call t a rotten driver? 



195. The corollary has been staled as a processing filter quite analogous to the competence filters of [Chomsky and 
Usnik77]. Fillers are a convenient method of describing the facts, but they are probably inadequate as explanations. In 
this case, wc cannot explain why lasl-resort seems to be the unmarked case. 

1%. There arc at Icasl three productive "counter-examples" to the corollary where the filter is inoperative. Wc will turn 
to these cases soon. 

197. The marked interpretation is excluded from certain dialects. 
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(426) + What do I consider John I? 

- Who do I consider t a fool? 

(427) + What did I tell the boy t? 

+ Who did I tell the story to t? 

- Who did 1 tell t the story? 

The last-resort strategy is consistent with die Tracc-X, Fitycr 1428k which is similar to constraint (429)-^ >lT»e 
constraint predicts that a trace of category X cannot appear just before lexical material of category X. 
Sentences (424)-(427) arc consistent with this gcncrali/^rtioivofthcTrac^^bWwHary. tJnfortunately, mere 
is little evidence in Hnglish to justify the move away front the Trace-NP Corollary, fllic crucial evidence 
comes from French.) 

(428) lbs tracc-X Filter : In the unmarked case, #{... tj X ...], where tj is a trace of category X. 

(429) The X X Kx traction Constraint : I fat some point in its derivation a sentence contains a sequence of 
two constituents of the same formal type, cither of wluch awld oc movtid or deleted by a 
transformation, the transformation may not apply to the first constituent in the sequence. 
(Hankamcr73J. 

Although the last-resort strategy has many of the right characteristics, there arc also many problems which 
require marked rules. Wc will consider the following three problems here: 19 ' 

(430) Ambiguity 

(431) lexical Marking 

(432) Length 



198. Hankamcr proposed lhal the XX Extraction Constraint belongs in competence. Since it can be violated (in the 
marked case), wc prefer lo place it in performance. [Fodpr78] also views the constraint as a processing matter. . 

199. It has been suggested that cleft sentences like. What I wanted that for I nobody could understand, form another class 
of marked exceptions lo the performance filter. 
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83.1 Ambiguity 

There arc some ambiguous sentences which strongly resemble the psqudo-attachment case. In the 
pseudo-attachment case, there is a lexical xp- with two possible mothers. PScudQ-gap is exactly analogous 
except the xp- is a trace. 

(433) Put the block in the box on the table. pseudo-attac/wtent 

(434) Who do you want (t) to cat (t)? pseudo-gap 

(435) The duck is too old (t) to cat (t). 

(436) Who did Mary promise (t) that she would marry (t)? 

(437) Po whom did Father say (t) mat he was planning to write (t)? 

(438) Where did he say (t) he was going (t# 

(439) When did he say (t) he was going (t)? 

Only (434) has been implemented, though the others shouldn't be much more difficult. l*scudo-gaps have 
many of the same problems as pseudo-attachment. It is (probably) impossible to find all the gaps in sentences 
like (440). YAP settles for the first and last possible gaps as in (441), in the absence of disambiguating 
information. 

(440) Who do you want (t) to want (t) ... (t) to want (t) to eat (t)? 

(441) Who do you want (t) to want ... to want to eat (t)? 

8.3.2 Lexical Marking 

The unmarked case can be overruled by tl»c lexicon as in (443). These cases have not bccnimplementcd. 

(442) + Who did the teacher walk to the cafeteria with? unmarked 
- Who did the teacher walk to tiie cafeteria? 

(443) - Which book did the teacher read to the children from? lexically marked 
+ Which book did the teacher read to the children? 



Lexical Marking 
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Kvcn though read and walk have the same subcatcgorization features (they both select an optional object and 
a verbal complement), they have different preferences as illustrated by (442) and (443). This evidence is often 
taken to support thd arc-ordering position. Although we accept fcxicatfy martcd preferences, there are other 
implications associated with that portion which arc incompalibte With the framework presented here; in 
particular, arc-ordering is crucially non-deterministic. 200 

8.3.3 length 

Notice that judgments arc less and less sharp as the second object increases in length. Iliis is completely 
unexplained by our account. Ilicrc are other length phenomena (such as heavy np shift) which axe more 
widely accepted. Wc seem to be missing a generalization. However it isn't flcar, how to capture the length 
phenomena. [Fra/icr and Fodor78] used a front end filter (PPP) which divided chucks into roughly six words. 
Although this is an interesting proposal.it isn't clear how it could Be implemented. • 



200. [Rich75] gives a critical review of the arc-ordering position. In his opinion: 

UnBuiflic Phenomenon ftmiwiiitiqntil Mechanism Assessment 

Center-embedding single-place HOLD list wrong 

inadequate 



Preferred readings of 
Ambiguous Sentences 

GP sentences 

Perceived Complexity 
Differences 



ordered trying of 
alternatives (arcs) 

back-tracking 

HOLD list costing 
arc counting 



somewhat right 



inconclusive 



His arguments arc very convincing. One could view YAP as a Dfi> which only backs up after it lakes a very serious GP. 
(We haven't implemented a GP recovery procedure yet. but backup would be the easiest way to do so.) A sentence is 
unacceptable just in case it causes YAP (as modified) to backup. This is a precise definition. The problem with the 
arc-ordering position is that backup describes both crashingly unacceptable GPs and extremely subtle preferences of 
ambiguous sentences. The sharpness is not related to any measure of backup dial has been proposed. Wc suggest that 
subtle preferences have a very different explanation from GPs. 
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(444) # Who did you call tit? 
???Who did you call t that? 
??Who did you call t a rotten driver? 
?Who did you call t the worst driver that you ever . 



8.-1 Summary 

Wc have discussed four cases of wh-movement: wh-qucstions, embedded questions, relative clauses, and 
topicalization. Movement constructions suggest some interesting topics in both competence and 
performance. 

(445) Competence: locality principles & island phenomena 

(446) Performance: gap finding 

We have shown that "unbounded" movement phenomena are local using an appropriate representation, such 
as Rrcsnan-Kaplan's fstructure. 201 Locality is extremely convenient for processing because it enables YAP to 
apply movement rules without approximation. If the rules were truly non-local they would require 
unbounded memory and hence wc should expect to discover empirical discrepancies from the competence 
idealization. However, since the idealization is local, there need not be any empirical discrepancies. 

The locality issues are extremely complex; we have only addressed a few cases. Much of the linguistic 
discussion deals with islands which are opaque to wh-movement. These islands should have a natural 
formulation in our representation (fstructure or move-alpha*). We have given an account (more or less) for 
two types of islands: wh-islands and Ross' Complex NP Constraint. This is still an active area of linguistic 
inquiry. 



.701. ll also is possible to represent movement locally in Chomsky's framework, using equivalence classes. We have 
previously suggested lhal Ikesnan-Kaplan's merge operator ( = ) is an equivalence relation. All the nodes which have 
been merged together (co-indexed) form a single equivalence class (index), which is represented as a single node in 
fstructure. For example, in the raising ease (move-up), all the understood subjects are co-indexed into a single node in 
fstructure. Similarl) co-indexed traces in comp (/iv/j in YAP) are also a single node in fstructure. 

Using the same basic approach, we could represent movement locally in Chomsky's system, let move-alpha be a 
relation between two phrases, and let move-alpha* be the transitive, symmetric and reflexive closure of move-alpha. 
Move-alpha* is similar to liresnan-Kaplan's merge ( = ) operator: it loo defines equivalence classes corresponding to the 
index. The claim that movement is local in tslmclurc corresponds to a claim that movement is local on indexes 
(equivalence classes under move-alpha*). 
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The most difficult problem is finding the gap. We have argued for a last-resort model. It is consistent With 
some garden path data and Hankamcr's XX Extraction Constraint, although it docs have some problems. The 
most serious problem is lexical marking. It was suggesjed $at marked mU$ c$ijd,appjy ,|n the crucial cases, 
although the proposal has not been implemented. There also appear to be some length effects, which arc also 
unexplained. We outlined a partial solution to the pseudo-gap phenomena. 

Despite these problems, we have implemented a simple device which captures many of the wh-movement 
phenomena. This result™ 'considerably weakens the traditional view that processors must be Turing 
Equivalent. The next chapter witt illustrate a "simple" mcchalism For parsing many conjunction phenomena, 
which were also believed to require inordinate resources. 



202. Many other researchers have designed "simple" devices to capture wh-movcmenl See [Marcus79J and 
[Gazdar79a,b.c] for two examples. 
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9. Conjunction 

Conjunction has been one of the most difficult constructions ip parse bqe^use there seem to be so many 
possible alternatives. Conjunction is a very good test of the FS hypothesis. How can we approximate the 
idljHtompctcncc model so mat ial^S processor can parsc^c6^Un<;Uont'Wvemadc rt s<>mc impressive initial 
progress, although there is sUlf subsiafiflai work io ikf'dolne. u ' s 

9.1 Simplifying Assumptions 

Many parsers have found conjunction difficult because they consider too many possibilities. It is extremely 
important to consider as Ifcw. altcrnajtivcs aSjpqssible, JNfi wil) impose several very strict limitations on 
conjunction in order to limit the scope of the problem. All of these restrictions arc controversial. 

9.1.1 The Constituent Assumption 

(447) Assumption : Conjunction applies to constituents, not to arbitrary fnujmcnts. 

(448) The scene [of the movie] and {of the play] was in Chicago. 
Which [boys] and [girls] went? 

[Which boys] and [which girls] went? 

Whidh boyslwcnt to the baflff and (took die jar]? 

Although (447) is generally accepted, there have been some objections. Sentences like (449)-(450) have been 
used to argue that conjuncts may not always M constituents. Wc will argue (hat despite appearances both 
(451) and (452) arc constituents. 



(449) John [drove through] and [completely demolished] a plate gla^s window. [Woods73] 

(450) Mary [expressed costs in dollarsj and [weights in pounds], ; ? f ! {MartinSQJ 

(451) [ vp drove through [ np .J] 

(452) [ [ v ] weights in pounds] 

The constituent assumption is very convenient for processing, as we will see. 
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9.1.2 The Category Assumption 

(453) Assumption : Fach conjunct has tlic same category . 

This assumption is also fairly standard, though there have J$cn arguments to the contrary. IMartinSQ] 
provides the following "counter-example". (455) is hjs analysis; (456^ i$,owr own* 

(454) Wc expect difficulties now and in the future . 

(455) Wc expect difficulties [ now] and [ in the future]. Martin 

(456) Wc expect difficulties L p. now] and £1. in the future]. YAP 

In this case, it seems reasonable to call now a prepositional phrase. This is a small cost to pay to save the 
category assumption. 

9.1.3 The Across-the- Board Convention 

(457) Assumption : Each conjunct has the same number of wh-£aps. Furthermore, flic gaps have the 
same category. 

The last three assumptions can be summarized in Gazdar-Notation 20 * as (458). 205 (The comparative 
construction illustrates the need for some more categories, (q, qp and 9P") to represent quantifiers. 
Comparatives have not been implemented.) 

(458) Assumption : Kach conjunct has the same Gazdar-Notation. 

(459) *John is easy [ vp ./np- to P' easc l and ] to love Mary]. 
John is easy [ vp . /np . to please] and [ vp . /np . to love]. 

(460) *Thc man who [ s/ Mary loves] and [^ Sally hates George] corrtpated my tax. 
The man who [ s/np . Mary loves] and [g/ n p_ Sally hates] computed my tax. 

(461 ) The kennel which [ s / np . Mary made] and I s / np . Fido sleeps in] has been stolen. 

The kennel in which [ s / pp . Mary keeps drugs and [g/pp. Fido sleeps] has been stolen. 



203. 1 1 seems that the gups have to be identical in every respect, not just category. That is, they have the same reference, 
person, number, gender, case, inflection, etc. 

204. In Ga/dar-Notalion, X/Y refers to a node of category X containing a gap of category Y. 

205. These examples are taken from [Gazdar79c]. Tough movement and comparative are not currently implemented. 
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The kegncKin) which [^/^Maiy mateJ^df^pwFkkksfcx^has been stolen. 

^L - . > 

. v ' ' } '• ' ■ k 

(462) Jolin saw more horses than [j/^,. Bill saw] or ^/ np . Pete talked to). 

John saw more horse? than Q^. Hill s^w ^pws^or ^^ t? P^ja^ to cats]. 
•John saw more horses than [^j.- Bill saw cows| or [g/ n p. Pete talked foj. 



9.2 Simple Cases 



In die simple case, the conjuncts happen to be in upl and down2 as below. 



(463) Bob and Bill saw it. 



(464) [ s Bob] 

Inp- **1 

= =WALL= = 

lconj and l 

Inp- Btt fl 
Iv^aw] 



first conjunct 



Second conjunct 



Conjunction is possible in (464) because downl is a conjunction and upl and dpwn2 are constituents of the 
same category with matching gaps. There is a marked rule which looks for this pattern. f . 

9.2.1 Attaching Conjuncts 



;'X/ 



Attaching conjuncts is different from other types of attachment; there is a special slot in cstructucc nodes for 
conjuncts. r 



(465) np- -> np- conj np- stanmfd ,. 

(466) [j [ np . [ np . Bob] and [ np . Bill]] ... 

(467) np- -> chead:np cxcomp:{ vp-} cxcomp:{s-} cconjuncts:np- YAP 

(468) [ s [ np . Bob and [ np . Bill]]... 

Using the standard approach, YAPeouldw't attach «o6 to the root because there migh*<bc a conjunction node 
in between. Consequently, attachment, wouldn't be possible unu| the right edge has been read. But this 
would prevent early closure <A-ovcr-A closure principle) because no nodccwuld be attached until air of its 
descendants have been completed. This is very unfortunate. YAP's approach avoids tin's problem because 
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there arc no nodes between the first conjunct Z?o6 and Ac toot, and hence, attachment is possible before 
conjunction is considered. 206 

After attaching the conjuncts, [ np . fiiftj wilt filllheccoi^MHC/ssfcit or( n pi Bobf and the iiiachinc state will be: 

(469) [ s Hob and Bill] 

Inp- Bi »] 

= =WALL= = 



'punct •' 
The sentence will now be parsed as if [ n _ Bill] is the subject 



207 



9.2.2 Attention Shift 

r lTic approach just outlined works on (470), but fails on (471) where minimal attachment is misleading. 



Fig J 4. Attention Shift 

sentence: I saw Bob and Bill saw me. 

input pointer: me. 

before after 

[j I saw Bob] [g I saw BobJ 

[ vp saw Bob] { vp saw Bob] 

Inp- **1 Inp- **1 

= =WALL== lconj and ] 

Uy^l ==WALL= 

Inp- Bi »l Inp- Bi »l 

[ v saw] L saw] 



206. Yet another alternative would use the standard phrase slmclurc rules. It would attach the first conjunct as if there 
were going to be a conjuration The ^^ {This may 
be a notalional variant of the current implementation.) 

207. There is only one difference; f np . Bob and frill] is plural whereas [ n _I HHrj U stflgirtar. YAf*Ts Solution assumes that 
all of functional features are inherited, in fact, number vfrtws arc tKttiiwScnted in the usual way. YAP actually replaces 
the number value in this case. There is a more attractive solution to be found. 



Attention Shift 



-m- 
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(470) 1 saw Rob and Bill. 

(471) I saw Bob and Bill saw me. 

ITie solution is to shift the attention 208 of the machine past the a/tt/biyldjng /W/ saw me bottom up. Then the 
machine will return its attention back to the conjunction and finish the sentence as if [ s Bill saw me) came 
prepackaged as a single unit 

YAP shifts its attention by moving downl into the upper buffer as in figure 14. Attention return is just the 
inverse; YAP moves upl back into the lower buffer. 200 I*e technique is very general; it allows bottom-up 
chunks to appear prepackaged. Attention shifting is heavily *ised to parse noun phrases. 

There are some important issues concerning the order of attention shift, and return in the default ps rules. 
Return is last. It isn't clear where shift should be; Marcus ordered it first, 210 we've ordered it much later. The 
issues arc not well understood; we're not prepared to make a ddherent argument. 



(472) IAP 

attach 
predict 



attention shift 
close 
find gap 
attention return 



Marcus' Parsifal 

attention shift 
find gap 
attach 
predict 
close 
attention return 



208. The terminology is Uiken from [Marcus791 who tiscd a similar technique to parse noun phrases. 

209. There are two registers associated with each node {as-slatm and as-remrn-status)»/\\kWpK\M infinite attention 
shifts and returns. The details aren't very interesting. 

210. Marcus" attention ,sl"ft mechanism -.was conditional on calcguB-y lypA,, Parsifal would attention ..shift for noun 
phrases, but not for verb phrases or prepositional phrases. Our mechanism applies to all categories. 

211. The ordering of actions in Martin Parsifal is partly dermcrf ty the itaemreUir (attehliofcsMft and return) smd partly 
implicit in the grammar (attach, predict, close and find gap). The implicit order may be incorrect; it is our own 
interpretation of his grammar. 
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9.2 J Closing 

After attention shifting to parse [ s Bill saw mc], the machine state is (473) (left side). The machine will then 
close upl repeatedly until conjunction is possible. 

(473) before after closing once after closing twice 

( s I saw Bob) 

[ saw Bob] [ s I saw Bob] 

[ np -Bob] f vp sawBob] [,,1 saw Bob] 

= =WALL== raWALLas rsWALUn 

lconj and l Uj and l Uj and l 

■J U M 

[ n p-« iU J Inp-^J VK* 

[ v saw] ( v saw] ] v saw] 

Conjunction applies just as it did in the simple case, / saw Bob and Hill. Down2 fills the cconjuncts slot of 
upl, leaving the machine in (474). The rest of the sentence parses just like the simple sentence, Bill saw me. 

(474) I,] 

= =WALL= = 

[ np .Bill] 

Lsaw] 



9.2.4 Summary of the Simple Cases 

let us summarize the simple conjunction rule. First, the machine attention shifts for the non-minimal 
attachment case (476). In the non-minimal case, YAP will predict an s just before Bill saw me. Then YAP 
will return attention to the and. 

(475) I saw Bob and Bill. 

(476) I saw Bob and Bill saw me. 

Secondly, YAP tries to attach conjuncts, if possible. Upl and down! have to be constituents and should 
match in category and gaps. Finally, if that doesn't work, YAP will close upl. 
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(477) Attention shift 

(478) Attach-conjuncts 

(479) Close 

ITiis approach has some problems. It finds only the lowest attachment, not the full range of ambiguous 
possibilities. YAP should pscudo-attach conjuncts in ambiguous cases such as (480). It should be possible to 
implement pseudo-attachment in these cases, but the details have not been worked out. 

(480) Bill told ftob [that Mike told Harry] and [Sam told Jackj. 

'llicrc arc more difficult cases where pseudo-attachment is not a likely solution. It is not clear how (482) and 
(483) could be represented in a single structure. Even worse, YAP prefers the unlikely interpretation (483) 
because Bill /?/?- builds a clause bottom-up. .>>?.■■•■-■ 

(481)1 know Bob and Bill left. 

(482) I know [Bob and Bill] left 

(483) [1 know Bob] and [Bill left] 

The general approach has been very effective although there are many problems to be solved. 

9.3 Deletions 

It is possible for one of the conjuncts to contain a deleted element. In the gapping case, the verb in the second 
conjunct is deleted; in right node raising, an object in the first conjunct is missing. 

(484) Bob saw Bill and Sue Mary. gapping 

(485) Bob looked at and Bill took the jar. right node raising 

Both of these constructions appear to violate the constituent assumption. With a deletion analysis, though, it 
is possible to save the constituent assumption. As we have suggested, (484H485) will be analyzed as: 212 



212. Right node raising is usually analyzed as: 
[Bob looked at lj] and [Dill Look t|] [the jarjj 



Our analysis is simpler to implement. Although this ;ilonc isn't a valid reason to prefer one analysis over another, there is 
sufficient controversy over right node raising that it didn't seem worth the effort to implement it precisely. 
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(486) [Bob saw Bill] and [ s Sue [ y ] Mary] 

(487) [Rob looked at [ ]] and [Rill took the jar] 



9.3.1 Right Node Raising 

YAP has a marked rule to parse right node raising. When there is a conjunction (e.g. and) in downl and upl 
can't close, then YAP assumes right node raising. The analysis crucially depends on the constituent 
assumption; if a conjunct is not a complete constituent, then by assumption the rest must have been deleted. 
Having detected the deletion, YAP undoes the transformation, inserting an empty noun phrase back into the 
buffer as in figure 15. 

The analysis has some problems; it docs not bind the empty noun phrase to an object in the second conjunct. 
YAP would erroneously accept ill- formed sentences such as (488). There is some controversy over die 
appropriate binding mechanism; it isn't clear if it is movement as in [Gazdar79c] or anaphoric. 213 



Fig. 15. Right Node Raising 

sentence: Rob looked at and Bill took the jar 

before after 

[ s Rob looked at] [ s Bob looked at] 

[ looked at] [ looked at] 

= =WA1.L== ==WALL= = 

[conj and l lnp-1 

tnp- >«»] Uj a » d l 

[ v took l [„ p -»ill] 

L took] 



213. It is generally agreed that the subject of drink is anaphorically bound in the following cases. 

Drinking gin can be fun. 

It doesn't require a glass to drink gin. 

Having drunk gin all day, I was completely wasted. 

There are several important differences between anaphoric control and movement. This paper though will not discuss 
bound anaphora. 



Right Node Raising 
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(488) *I took and you went 

Optional arguments illustrate another problem. YAP will detect only obligatory elements which have been 
deleted; optional elements are also^ubject to deletion. YAP-wM not detect an object of ate in (489*. 

(489) 1 ate ([ ]) and you drank everything they brought 

9.3.2 Gapping 

"Gapping" is the case where the second conjuncts verb has been deleted. (490) is a simple example. 

(490) [Bob saw Bill] and [ s Sue [ v J Mary] 

YAP parses these by undoing the transformation. When the lower buffer contains a conjunction followed by 
two noun phrases, YAP inserts an empty verb into the buffer. T<*cxch*dc interpretations such a&(492), YAP 
merges the predicates from both conjuncts. See figure 16. 



Fig. 16. Gapping 

sentence: Bob saw Bill and Sue Mary. 

[ s Bob saw Bill] be f ore transfoimalion 

= =WALL= = 

Uy and l 

Wr SUC1 
[ np .Mary] 

[ s Bob saw Bill] a f ter transformation 

= =WALL= = 

Uy and ] 

Inp- Sucl 

[ ] sec-1 -> fsubj:np- fobj:np- 

[ np .Mary] 
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(491) Bob persuaded Bill to leave and Sue Mary. 

(492) *Bob persuaded Bill to leave and (Bob persuaded) Sue Mary. 

The implementation is not as general as it should be; the vert* cajj be deleted in many other contexts. YAP 
can find a deleted verb in any projection of v (in vp, vp-, s and s-). For example, YAP correctly parses (493). 
Unfortunately, it finds only the lowest possible inter^fctiitkJir; it will hbt diScbvcr{494) unless there k some 
positive reason (i.e. semantics) to reject (493). The gapping pattern crucially depends on two noun phrases; it 
will not detect gapping when the second object is an xcomp as in (495M497). Aside from the ambiguity 
problem, these problems shouldn't be too difficult to correct. The simple cases of gapping were implemented 
to show plausibility -within our n&tritted framework. 

(493) Bob [gave Bill a ball] and [ vp Sam a jarj. 

(494) [Bob gave Bill a ball] and ^ Sam ajar.] 

(495) lk)b persuaded Bill to leave and Sam f v _ to stay]. 
(4%) 1 expressed costs in dollars and weights f ia pound*], 
(497) I considered Bill likely to win and Sam [ likely to lose]. 

9.4 Summary 

In summary, we have presented a simple approach to parse many conjunction constructions including some 
cases of right node raising and gapping. Although there arc many problems to be solved, these analyses 
indicate that it is plausible for a FS deterministic processor to parse conjunction. This discussion responds to 
the traditional arguments that a FS processor cannot in principle capture the conjunction generalizations. 

We have previously suggested that closure actually simplifies conjunction. YAP uses closure to find the first 
conjunct: it will continuously close off upl until the first conjunct is in up I. Furthermore, closure assures that 
all /joss/We conjuncts will be in the upper buffer; this makes it much easier to pscudo-attach conjuncts siriee it 
is easy to find ;ill the possibilities. 214 



214. YAP docs not currently pscudo-attach conjuncts, although it was designed with this in mind. 
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10. Conclusion 

Wc have hypothesized that a computationally simple device is sufficient for processing natural language. By 
incorporating two processing constraints, FS and Marcus' l^mmism, it was possible to construct a parser 
which approximates many competence idealizations. YAP was designed to foil precisely where the 
idealizations require unrealistic resources. YAP's success, as far as it goes, provides some evidence for the 
hypothesis. 

10.1 The Traditional Position 

Traditionally there have been many arguments for computationally complex models of natural language. 
Much of the early literature, though, docs not refute our hypothesis, but merely cast doubt on its feasibility. 
Admittedly, it is easier to find descriptions using more powerful (complex) techniques, but is it necessary to 
use more powerful techniques? The traditional arguments are extremely negative; if the problem is really as 
hard as they suggest, then the only solution is to grin and bear it. It is easy to show how hard a problem might 
be, but it is a real accomplishment to find a simple elegant solution. 

Chomsky's early arguments arc rightly cautious; they do not exclude the possibility of a FS processor. He 
criticizes contemporary FS approaches as ifl£lqganj. and then p«^^computat%nally complex alternative 
as more revealing. Over the years, however, his position has been misinterpreted as a complete refutation of 
FS approaches. It is merely a feasibility argument. To a certain extent he is correct, [Chomsky56, pp ; 413] 
"the grammar of Knglish is materially simplified if phrase structure description is limjtedto akemel of simple 
sentences from which all other sentences arc constructed WW**®* transformation; and that this view of 
linguistic structure gives a certain Height into the, use and wndefstandw* 9? language." Hence, competence 
idealizations should use powerful device* However, this docs nol say that language tfiould 4* processed by 
exactly the same machinery. 

This is a very common situation in engineering. Engineers develop ideal models to gain fruitful insights; they 
do not expect their model to perfectty «*plicatc the real world. They will us* the theory as far as it goes, and 
then joke about "Murphy's Law"'.' idealizations are •y^i^^M^c^ibo taken too seriously; ihey 
simply don't work in all cases. Physical machines do not bdwveidcaiiy^ 
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[Chomsky56] provides a "counter-example" to FS models. It generates arbitrary ccntcr-embcdding and 
hence it is beyond the generative capacity of a FSM. Since his counter-examples arc grammatical (part of the 
ideal competence model of language), this proves that a FSM cannoi process competence/ 215 However, it is 
well-known that arbitrarily deep ccntcr-embcdding is urn vcrsally unacceptable, and hence, Chomsky's 
arguments do ml apply to performance. We have no reason to exclude dtc possibility of a FS parser. 

He correctly suggests that a parser should encode a simple and "revealing" grammar. It is not clear how this 
can be accomplished with a simple device. YAP introduces a number of approximations (i.e. bounded stack, 
finite lookahcad, ...) in order to approximate an elegant (though complex^ competence gramniar With 
reasonable resources. Chomsky has questioned this move for two reasons: 216 



215. "Turning now to English, wc find that there arc infinite sets of sentences that have dependency sets with more than 
any fixed number of terms. For example, let S^ S 2 , ... be declaraiive sentences. Then the following arc all English 
sentences: 

(13) (i) IfSj.thenSj. 
(ii) Either S3, or S 4 . 
(iii) The man who sakl that Sj. is arriving today. 

These sentences have dependencies between 'if-'uW. 'eithcr v or\ man'-"is\ Botwe can choose Sj.Sj, S 5 which appear 
between the interdependent words, as (13i),(13ii), or (I3iu) theniselvss." {ehom*ky56 pp, 115J 

216. "Although we have found that no finite-stale Markov process [YAP] that produces sentences from left to right can 
serve as an Fnglish grammar [competence], we might inquire into the possibility of coftstriictfog a sequence of such 
devices that in some nontrivial way, com- closer and closer to matching the output of a satisfactory Fnglish grammar. 
Suppose, for example, that for fixed n wc construct a finite-slate grammar in the following manner: one slate of the 
grammar is associated with each sequence of English words of length n (ordered by statistical frequency] ... as n increases, 
the output of such grammars wilt come to took more and more llfceiRnglii^^. ;Tbis fact has occasionally led lathe 
suggestion that a theory of linguistic structure injglu be fashioned on such a model ... 

Whatever the other interests of statistical approximation in this scnsVmay' be, it is clear that it can shed no light oh the 
problems of grammar. There is no general relation between the -frequency of aString.(or its component parts) and its 
grammaticalness ... there is no significant correlation between order of approximation and grammalicalness. If we order 
the strings of a given length in terms of order of approximations to Fnglish, we shall find both grammatical and 
ungrammalical strings scattered throughout the list, from lop lo bottom. Hence the notion of statistical approximation 
appears lo be irrelevant lo grammar." [Chomsky56 pp. 1 16] 
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(498) Arc the approximations revealing? 

(499) What arc reasonable approximations? 

We have attempted to respond to both points. First, they are revealing because they suggest a number of 
crucial differences between competence and performance. For example, Lasnik's Noncorefcrcnce Rule is an 
impractical idealization; a more realistic approximation (using the A-ovcr-A early closure principle) predicts 
certain preferential possibilities which may actually reflect the real empirical facts more accurately than 
Lasnik's idealization. We have discussed many other constructions which arc similar in this respect, such as: 
center-embedding, crossed dependencies and garden paths. 

Chomsky's second criticism is also well-taken; it is very difficult to find independently motivated 
approximations. He rightly criticizes a statistical approach for missing the relevant generalizations. In this 
work, we have attempted to motivate effective approximations without sacrificing linguistic significance. 
YAP captures many linguistic generalizations such as: raising, passive, there-insertion (chapter 6), inversion, 
imperative (chapter 7), wh-movement (chapter 8), and conjunction (chapter 9). 217 These generalizations are 
basically orthogonal to the two processing approximations: FS and determinism. Hence, the approach taken 
here may be a reasonable compromise between processing complexity and linguistic elegance. 

10.2 Summary 

We have been most concerned with two performance constraints: FS and determinism. Both of these 
constraints reduce the computational power, which is always a welcome step in computer science. The 
question is whether the machine retains enough oowcr to parse language. We have demonstrated, by 
implementing YAP, that it is sufficient to parse certain difficult constructions. Furthermore we have 
defended a number of simplifying assumptions as more accurate descriptions of the empirical facts. 
Chapters 1 through 4 discussed some evidence involving center-embedding, crossing dependencies and 
noncorefcrcnce. These constructions arc provably complex (in competence), and as predicted, they do not 
behave ideally, even at severely shallow depths. This is suggestive evidence in favor of our simplifying 
assumptions. It appears that all examples of complex behavior arc universally unacceptable. 



217. One could rightly criticize these transformations as mere stipulations. A truly revealing theory would explain the 
facts. We have described (stipulated) many of Bresnan-Kaplans analyses as they are. When deeper explanations are 
found, it may be worthwhile lo redesign YAP. 
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llicrc arc many difficult issues dealing with a particular implementation of the approximations. Chapter 2 
discussed several closure proposals. We finally settled on a compromise (the A^ovcr-A early closure principle) 
which lias some of the right limiting properties (w.r.L premature/ineffective), but may have some problems in 
certain borderline cases (tiircc deep center-embedded sentences). The limiting cases arc far more important; 
the field may not have progressed sufficiently far to make the subtle distinctions necessary for the borderline 
cases. 

Chapter 4 dealt with attachment strategies. We advocated a default mode of operation (attach, predict, and 
then close) which covers most cases although there arc many exceptions. The exceptions rail into four classes: 
early closure (chapter!), non-minimal attachment (chapter 4), pseudo-attachment (chapter 4) and 
transformations (chapters 6-9). 

Pseudo-attachment illustrates the delayed binding approach which is a recurrent theme in this work. The idea 
is to avoid making decisions which may have to be taken back at a later time: This is particularly crucial in a 
deterministic framework which prevents the system from reverting previous commitments. In the 
pseudo-attachment case, the system can decide mat it cannot decide how to attach, and hence it attaches both 
ways. 

The delayed binding approach is also central to feature manipulation (chapter 5). An alternative approach 
would try each feature value combination non-detcrministically until it found a combination which doesn't 
violate any agreement constraints. This can be very time consuming. YAFs approach is a constraint 
propagation technique; it applies the constraints themselves to the (structure. The difference between the two 
approaches becomes apparent when the constraints undcrdctcrminc the final outcome, such as (500)-(501)- 
YAP makes a single deterministic pass; it is no harder to search an undcrdctcrmincd (structure than any 
other. A non-deterministic parser, on the other hand, has to search the (structure once for each combination 
of values; the undcrdctcrmincd case requires much more time because there are more combinations of values. 

(500) 1 pjjj it down. underdetermined tense 

(501 ) The deer left underdetermined number 

The lexicalist position is very compatible with a delayed binding approach. Although it is possible to write a 
deterministic transformational grammar (as Marcus did), we have found the lexicalist position more 
sympathetic with the notion of constraints, which is crucial in our formulation of delayed binding. For 
example, both approaches have a mechanism for "raising" understood subjects as in (502); Krcsnan and 
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Kaplan use the constraint equation, sub/up) - subj(xcomp(uf>)), where Chomsky uses the transformation 
move-np. Brcsnan-Kaplan's constraint equations fall rather naturally into a constraint propagation 
framework; it might require some ingenuity to reformulate Chomsky's movement as a constraint. Although it 
is probably possible to reformulate movement in this way, Brcsnan-Kaplan's formulation requires little 
modification to fit into a constraint propagation framework. 

(502) Joluij seems x { to be a nice guy. 

In summary, we have proposed that a deterministic FS parser is sufficient to parse natural language without 
sacrificing linguistic generalizations. To justify this claim, wc have designed yet another parser (YAP) which 
encodes many of Brcsnan-Kaplan's analyses in a deterministic FS framework. Although there arc many 
unsolved problems (i.e. lexical ambiguity, syntactic/semantic interaction,...), wc have demonstrated 
plausibility for the underlying design which incorporates both performance (FS and determinism) and 
competence (Brcsnan-Kaplan's lexical framework). 
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Appendix I - Some Results 

Sentences (503)-(536) were taken from Appendix D [Marcus79J. These examples illustrate passive, raising, 
there- insertion, some lexical ambiguity {thai, meet and schedule), auxin version, imperative and 
wh-movcmenL YAP can parse all of them except (534) which is unacceptable. Chapter 8 discusses this 
sentence in more detail. 

(503) I told that boy that boys should do it that diagnostic 

(504) The jar seems to be broken. passive, subject raising 

(505) There seems to be ajar broken. there- insertion 

(506) ( wanted John to do it 

(507) 1 want to do it . 

(508) I persuaded John to do it object rasing 

(509) There seems to have been a meeting scheduled for Friday. 

(510) Schedule a meeting for Friday. imperative 

(5 1 1) Is there a meeting scheduled for Friday? aux- inversion 

(512) A meeting seems to have been scheduled for Friday. 

(513) I told the boy that i saw Sue. 

(514) I told Sue you would schedule the meeting. 

(515) 1 told the girl that you would schedule the meeting. 

(516) The boy who wanted to meet you scheduled the meeting. wh-movement 

(517) liie boy who you met scheduled the meeting. 

(518) Who did John see? 

(519) Who broke the jar? 

(520) What did Bob give to Sue? 

(521) Who did Bob give the book? 

(522) Who did Bob give the book to? 

(523) I promised John to do it 

(524) Who did you say that Bill told? 

(525) You promised to give the book to John. 

(526) Who did you promise to give the book to? 

(527) Who did you promise to schedule the meeting? 

(528) Who did you say scheduled the meeting? 

(529) Who did you persuade to do it? 

(530) What did you give Sue yesterday? 

(531) Who did you ask to schedule the meeting? 
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(532) Who do you want to give a book to tomorrow? 

(533) Who did you want to give a book to Sue? 

(534) it I gave the boy who you wanted to give the books to the books? 

(535) Who did you promise to give the book to tomorrow? 

(536) Who did you promise to give the book to Sue tomorrow? 

YAP can also parse the following conjunction sentences. These sentences were selected to illustrate YAP's 
abilities, both positive and negative. Many of these sentences may be unacceptable and/or ungrammatical for 
reasons which YAP docs not consider. For example, YAP docs no pragmatic analysis; (540) is syntactically 
well-formed even though it may sound somewhat odd. Similarly, (541) is probably ungrammatical because 
the trace has conflicting case; it receives objective case from the first conjunct and oblique case from the 
second. It would be simple enough to change the grammar accordingly. Finally, (542) demonstrates a real 
problem with YAP's formulation of right node raising; YAP does not require the missing noun phrase to 
"match" with the right most noun phrase in the second conjunct. Although there are some problems with 
YAP's formulation of conjunction, it demonstrates some real progress. 

(537) Which boys and girls went? 

(538) Which boys and which girls went? 

(539) Which boys went to the ball and took the jar? 

(540) Which boys went to die ball and into it? 

(541) What boy did bill look at and give a ball to? 

(542) lk)b looked at and gave a ball to the boy. 

(543) Dob gave Hill a ball and John a jar. 

(544) Hob saw Rill and Sue Mary. 

(545) I want Hill, Rob, and John to be nice. 

The following sentences were taken from a homework problem given by Ken Hale last fall. The first set are 
all grammatical; the second violate island conditions and, hence arc ungrammatical. YAP can parse all the 
grammatical ones and none of the ungrammatical ones. Sec the discussion of island phenomena in chapter 8. 

(546) Who should 1 ask where I can get a copy of Aspects? 

(547) What is it expected that Max will work on next? 

(548) What do you expect that Max will work on next? 

(549) What is Max expected to work on next? 

(550) What do you expect Max to work on next? 

(551) Who is expected to work on case-marking next? 
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(552) Who saw what? 

(553) I wonder who bought what 

(554) John wonders where to put what. 

(555) John wonders what to put where. 

(556) What does John want to put where? 

(557) Where docs John want to put what? 

(558) Who did you And a photograph of? 

(559) It is believed John has won Ac election. 

(560) John is believed to have won the election. 

(561 ) *Wlio docs John wonder where Bill saw? 

(562) *What did you ask me where you could buy? 

(563) *What is expected that Max will work on next? 

(564) *What is expected Max to work on next? 
(565)*Whatdidwhosee? 

(566) *l wonder what who bought? 

(567) *What does John wonder where to put? 

(568) *Whcrc docs John wonder what to put? 

(569) 'What docs John wonder to put where? 

(570) *Whcrc docs John wonder to put what? 

(571) *Who do you know the man that married? 

(572) *Who did you hear a rumor that John betrayed? 

(573) *Who did you find a copy of a photograph of? 

(574) *John is believed has won the election. 

(575) *John seems won the election. 

The following illustrate some other generalizations: 

(576) It seems likely that John would be sitting. it-extraposition 

(577) There seems to be a table in the kitchen. there- insertion 

(578) That I might take a ball seems likely. sentential subjects 

(579) For mc to take a ball seems nice. 

(580) To take a ball seems nice. 

(58 1 ) I wonder what to do. embedded questions 

(582) I wonder what he should do. 

(583) 1 wonder what should have been done. 
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(584) The ball, he took. topicalization 

Wc have said very little regarding lexical ambiguity, although there arc a few marked rules to cover some 
simple cases. There is one rule to distinguish an auxiliary from a main verb and another to separate the 
various uses of that (a complementizer, a relative pronoun, a normal pronoun, and a determiner). The first 
rule was discussed in chapter 7. Neither rule is particularly elegant; Milne is working on more attractive 
solutions to the lexical ambiguity problem. 

(585) Have the boys take the ball! auxiliary diagnostic 

(586) Have the boys taken the ball? 

(587) Which boys were the girls taking to the ball? 

(588) Which boys have the girls take the jars? 

(589) Which boys have, the girls taken to the ball? 

(590) I know a man that was nice. that diagnostic 

(591) I know that was nice. 

(592) I know that that was nice. 

(593) I know that boys arc nice. 

(594) I know that boy is nice. 

(595) I know that he is nice. 
(5%) That he is nice is a fact. 

(597) That that boy is nice is a fact 

(598) That that is nice is a fact 

(599) Who do you believe that was? 

(600) Who do you believe that that was? 

(601) Did you believe that? 

(602) Did you believe that was him? 

(603) Did you believe that that was him? 

(604) Did you believe he did that? 

We discussed pseudo-attachment briefly in chapter 4 and pseudo-gaps in chapter 8. (605), (606) and (608) 
illustrate the phenomena; (607) and (609) arc near misses (they have only one attachment/gap). 
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(605) He seems nice to her. pseudo-attachment 

(606) Put the box on the table in the kitchen. 

(607) Put the box on the tabic. near miss 

(608) Which boys docs he want to see? pseudo-taps 

(609) Which boys docs he want to take? near miss 

Wc have been very concerned with stack allocation. (6I0H6I2) illustrate some borderline center-embedded 
sentences. 218 YAP docs require one less stack cell For (610) than the others, although the reason is very 
complex. Wc don't have enough confidence in the details to trace though the entire explanation. The 
generalization seems to be that a complement is less acceptable in the most deeply embedded clause 
[Cowpcr76 pp. 71]. YAP finds deeply embedded complements more difficult because it is hard to distinguish 
them from relative clauses without storing the entire sentence on the stack. 

(610) The possibility that the man who I hired is incompetent worries me. 

(611) #'l"he man who the possibility that students arc dangerous frightens is nice. 

(612) #The man who the possibility that I am dangerous frightens is nice. 

YAP can also parse the following right branching sentences. (616) is somewhat problematic because die two 
that's arc disambiguated in the wrong order. Hence L_. of /J is attached to rumor. These diagnostics are not 
well understood. 

(6 13) It might seem likely that it would seem likely that he is nice. 

(6 14) 1 )id you hear a rumor that there was a possibility that he might say that I am nice? 

(615) Did you hear a rumor that there was a possibility that he might tell me? 

(616) Did you hear a rumor that there was a possibility that he might tell me of? 

(617) Did you hear a rumor that there was a possibility that he might tcHme of it? 

(618) Did you hear a rumor that it would seem likely that he is nice? 

(619) Did you hear a rumor that John wondered who said that I am nice? 



218. The first two are taken from [Cowpcr76j. 
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This appendix shows the derivation of (620). The final output (the (structure and the {structure) arc given as 
(621) and (622) below. 219 



(620) Was the boy likely to sit? 

(621)CSUBJ:[(NP-)theboy] 
CHEAD: [(NP) the boy] 
CSPEC: [(DET) the] 
CHKAD: [(N) boy] 
CHKAD: [(VP) was likely to sit] 
CHEAD: [(V) was] 
CXCOMP: [(AP-) likely to sit] 
CHKAD: [(AP) likely to sit] 
CHEAD: [(A) likely] 
CXCOMP: [(VP) to sit] 
CCOMP: KCOMP) to] 
CHEAD: [(VP) sit] 
CHEAD: [(V) sit] 

(622) FSUBJ: [(NP-) the boy] 
FSPEC: [(DET) the] 
FXCOMP: [(AP-) likely to sit] 
FSUBJ: [(NP-) the boy] 
FSPEC: [(DET) the] 
FXCOMP: [(VP) to sit] 
FSUBJ: [(NP-) the boy] 
FSPEC: [(DET) the] 



cstruclure 



/structure 



sentence: was the boy likely to sit? 



initial slate 



219. This source was produced by a slightly older version of YAP. Nevertheless, it should still be highly informative. 
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input pointer: LIKELY TO SIT ? 

I(S)] 

= =WALL= = 

[(V)wasJ 

[(DKT) the] 

l(N)boy] 

YAP will undo the inversion, but first it has to parse [ np _ the boy] to trigger the marked inversion rule. This is 
accomplished by the rule: APPLY-DEFAULT-AITKN TION-SHIHT. No rule of higher priority can apply 
because up I is looking for a subject, not a verb. 

input pointer: TO Srr? 

l(S)] 

[(V) was] 

= =WALL= = 

RDET) the] 

I(N)boy] 

[(A) likely] 

The determiner in downl triggers a marked rule to predict a noun phrase: CREATE-NP-1. 

input pointer: TO Srr? 

KS)] 

[(V) was] 

= =WALL= = 

l(NP)l 

[(DET) the] 

[<N)boy] 

[(A) likely] 

The NP is attention shifted to allow Lj ct the] and [ n boy] to attach. The next three snap-shots show the 
attention shift and two attachments. 
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input pointer: TO SIT? 

[(S)] 

[(V) was] 

[(NP)] 

= =WALL= = 

[(DKT)thc] 

l(N)boy] 

1(A) likely] 

About to run: APPLY-DEFAUI/r-PS-A'ITACHMENT 

input pointer: SIT ? 

l(S)] 

[(V) was] 

[(NP)thc] 

= =WALL= = 

I(N)boy] 

[(A) likely] 

[(COMP) to] 

About to run: APPLY-DKFAULT-PS-ATTACHMENT 

input pointer: ? 
[(S)] 

[(V) was] 
[(NP) the boy] 
= =WALL= = 
[(A) likely] 
[(COMP) to] 
[(V) sit] 

Now [ the boy] has all of its children, but it doesn't have' a mother yet. It will be returned to the lower 
buffer, so it can find its mother. (Slightly contrary to the discussion in chapter 3, ps-closc docs an attention 
return if upl isn't ready to close. In this case, upl can't close because it doesn't have a mother. 
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About to run: APPLY-DEFAULT-PS-CLOSURE 

l(S)] 

{(V) was] 
=--=WALL= = 
[(NP) the boy] 
[(A) likely] 
[(COMP)to] 

llic NP in downl triggers a marked rule (CREATE-NP--1) 220 to predict an np-. which is immediately 
attention shifted, leaving the machine in the following state. Then ps-attach and ps-closc apply producing the 
next two snap-shots. 

l(S)] 

[(V) was] 

[(NP)] 

= =WALL= = 

[(NP) the boy] 

[(A) likely] 

[(COMP)to] 

About to run: APPLY-DEFAULT-PS-ATTACHMENT 

KS)] 

[(V)was] 

[(NP) the boy] 

= =WALL= = 

[(A) likely] 

[(COMP)to] 

KV)sit] 

About to run: APPLY-l)liFAULT-PS-CI.OSURE 



220. The rule CREATE-NP-I predicts an np- whereas the rule CREATE-NP-I predicts an np. 
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'lTicrc is nothing left to do but attention-return, hoping to trigger some other rule. In this case, it will enable 
auxiliary inversion. (It should have predicted an ap- first. This indicates a slight problem.) 

[(S)] 

[(V) was] 

= =WALL= = 

[(NP-)thcboy] 

[(A) likely] 

[(COM P) to] 

About to run: APPLY-DKFAULT-PS-CLOSURE 

[(S)] 

= =WALL= = 

[(V) was] 

{(NP-)thcboy] 

[(A) likely] 

About to run: AUX-INVERSION 

Now, ps-attach can apply. 

KS)1 

= =WALL= = 

[(NP-) tlie boy] 

[(V) likely] 

1(A) to] 

About to run: APPLY-DHFAULT-PS-A1TACHMENT 

Notice that [ the boy] was automatically closed, removed from the buffer, after it was attached. In this 
older version, the closure procedure was very much like Kimball's scheme. ITic current scheme would not 
close this early; it would leave the np- in upl and then ps-closc would apply. 
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[(S)thcboy] 

= =WALL= = 

[(V) was] 

[(A) likely] 

[(COMP) to] 

About to run: PRED-DEFAULT 

This rule selects the appropriate predicate for downl from the lexicon. 

[(S) the boy] 

= =WALL= = 

](V)was] 

[(A) likely] 

[(COMP) to] 

About to run: AITACH-FSUBJ 

There is a slight problem checking functional constraints with elements to tile left of the head (such as 
subject). Conscqucndy, they arc checked by a marked rule (ATTACH-FSURJ) which fires when upl has a 
predicate and a subject. (We are currently exploring more elegant alternatives.) 

[(S) the boy] 

= =WALL= = 

[(V) was] 

[(A) likely] 

[(COMP) to] 

About to run: CREATE-VP-1 

There is a marked rule to build verb phrases bottom-up. (It is probably unnecessary.) With a more 
symmetric default predict rule, it should be possible to eliminate most of the marked prediction rules 
(CREATE-...). 
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[(S) the boy] 

[(VP)] 

= =WALL= = 

[(V) was] 

[(A) likely] 

[(COM P) to] 

About to run: APPI.Y-DHFAUL'l'-PS-AlTACHMENT 

YAP finishes the parse using the same techniques. 

[(S) the boy was] 

[(VP) was] 

= =WALL= = 

[(A) likely] 

[(COM P) to] 

[(V) sit] 

About to run: CREA TE-XCOMP-1 

[(S) the boy was] 

KAP-)] 

= =WALL= = 

[(A) likely] 

[(COM P) to] 

[(V)sit] 

About to run: PR ED-DEFAULT 



An Example 

[(S) the boy was] 

[(AP-)] 

= =WALL= = 

[(A) likely] 

[(COM P) to] 

[(V)sit] 

About to run: AITACH-FSUBJ 

[(S) the boy was] 

[(AP-)] 

= =WALL= = 

[(A) likely] 

[(COM P) to] 

[(V) sit] 

About to run: CREATE-AP-1 



Notice that the ap- will close when the ap is attached in the next snapshot. 

[(S) the boy was] 

[(AP)] 

= =WALL= = 

[(A) likely] 

[(COMP) to] 

[(V) sit] 

About to run: APPLY-DEFAULT-PS-ATTACHMENT 
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[(S) the boy was likely] 

[(AP) likely] 

= =WALL= = 

[(COM P) to] 

[(V) sit] 

l( PUNCT)?] 

About to run: CRHATF-INF-VCOMP 

[(S) the boy was likely] 

[(VP-)] 

= =WAl.L= = 

[(COM P) to] 

[(V) sit] 

[(PUNCT)?] 

About to run: APPLY-DEFAULTPS-A1TACHMRNT 

[(S) the boy was likely to] 

[(VP-) to] 

= =WALL= = 

[(V)sit] 

[(PUNCT)?] 

About to nin: PRRD-DEFAULT 

[(S) the boy was likely to] 

[(VP-) to] 

= =WAI.L= = 

[(V) sit] 

[(PUNCT)?] 

About to run: A'lTACH-FSUBJ 
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[(S) die boy was likely to] 

[(VP-) to] 

= =WALL= = 

[(V) sit] 

[( PUNCH?] 

About to run: CREATH-VP-1 

[(S) the boy was likely to] 

[(VP)] 

= =WALL= = 

[( V) sit] 

[(PUNCT)?] 

About to nm: APPLY-DEFAULT-PS-A'ITACHMENT 

[(S) the boy was likely to sit] 
= =WALL= = 
[(PUNCT)?] 
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